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The  purpose  of  this  study  was  to  develop  a  set  of 
guidelines  for  improving  the  selection  of  administrators  of 
public  community  colleges.  Fifty-two  research  studies  were 
found  which  met  the  parameters  of  this  study.  Only  those 
studies  which  concerned  selection  of  administrative  personnel, 
and  were  published  in  the  United  States  from  1962  through 
1985,   inclusive,  were  used. 

Predictor  variables  consisted  of  nine  personnel 
selection  methods:  (a)  aptitude  and  intelligence  measures, 
(b)  personal  interviews,  (c)  job-related  skills  indicators, 
(d)  psychological  attribute  indicators,  (e)  value  systems 
assessments,  (f)  biographical  information,  (g)  peer  ratings, 
(h)    self-appraisals,    and    (i)    assessment    center  processes. 
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Outcome  criteria  consisted  of  several  specific  measures  of  on- 
the-job  achievement  in    administrative  positions. 

The  statistical  procedure  of  meta-analysis  was  applied 
to  the  128  validity  coefficients  found  in  these  studies.  The 
use  of  this  procedure  allowed  an  objective  look  into  cross 
sections  of  the  research  data.  For  each  cross  section  of  the 
data,  mean  correlations,  weighted  by  sample  size,  were 
computed.  Sampling  error,  the  difference  between  the 
population  parameter  and  the  sample  statistic,  was  corrected 
for  in  each  of  the  resulting  mean  correlations.  The 
reliabilities  of  the  independent  and  dependent  variables  were 
computed  according  to  reliability  data  reported  in  the 
research.  These  data  were  used  to  adjust  the  mean 
correlations      for     error     of     measurement.  Thus,  new 

correlations,  closer  to  the  true  score  correlations,  were 
derived. 

Major  findings  were  that  (a)  all  personnel  selection 
methods  considered,  with  the  exception  of  self-appraisals, 
were  valid  predictors  of  administrative  achievement,  (b)  the 
methods  which  showed  the  highest  validity,  overall  assessment 
ratings  and  job-related  skills  indicators,  were  the  least  used 
in  personnel  selection,  and  (c)  those  methods  used  in  the 
selection  of  community  college  administrators,  biographical 
information,  personal  interviews,  and  peer  ratings,  were 
moderately  valid.  Ten  guidelines  for  administrative 
personnel  selection  in  public  community  colleges  were  derived. 


Legal  constraints  placed  upon  personnel  selection  in  public 
organizations  were  considered.  These  guidelines  included  (a) 
establishing  assessment  centers  for  selection  and  training, 
(b)  increasing  the  job-relatedness  of  selection  procedures, 
and  (c)  using  statistical  methods  to  simulate  overall 
assessment  ratings. 
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CHAPTER  I 
INTRODUCTION 

The  purpose  of  this  dissertation  was  to  develop  a  set  of 
guidelines  for  improving  the  selection  of  administrators  of 
public  community  colleges.  The  research  method  used  was  a 
meta-analysis  of  administrative  personnel  selection  research 
done  in  the  United  States  in  the  past  24  years. 
Administrators  in  our  public  community  colleges  must  possess 
both  extensive  knowledge  and  sound  judgment  to  meet  the 
complex  challenges  of  the  years  ahead.  The  extent  to  which 
a  community  or  junior  college  meets  its  challenges  largely 
depends  upon  the  gualities  of  its  future  leaders.  Thus,  these 
integral  people  must  be  chosen  carefully  if  they  are  to  be 
properly  matched  to  the  demands  of  their  positions.  Those 
concerned  with  the  placement  of  administrators  in  higher 
education  can  no  longer  afford  the  luxury  of  using  arbitrary 
selection  methods.  This  is  true  in  times  of  surplus,  and  even 
more  true  in  times  of  shortage,  budget  cuts,  and  retrenchment 
(Mandell,  1964).  Rapid  change  itself  has  mandated  a 
continuous  reevaluation  of  personnel  selection  procedures. 
Administrators,  well-gualif ied  in  the  past,  may  not  be 
prepared  to  face  challenges  of  the  job  in  the  late  1980s  or 
1990s.    According  to  Sharp  (1984) ,  community  colleges  grew  so 
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rapidly  in  the  years  between  1960  and  1980  that  education 
writers  spoke  of  the  "community  college  movement"  (p.  12)  . 
During  this  time  enrollments  swelled  from  400,000  to 
4,000,000.  These  institutions  called  forth  new  and  different 
types  of  leaders.  As  Lynch,  former  member  of  the  Wall  Street 
Journal  news  staff  (in  Sharp,  1984),  stated,  "gone  are  the 
days  when  college  presidents  were  tweedy,  pipe-smoking  types 
who  ran  things  at  a  leisurely  pace  as  money  rolled  in  and  new 
buildings  were  rising  all  over  their  campuses"  (1983,  p.  61). 
These  changes  have  impacted  upon  the  levels  of  administrators 
subordinate  to  the  college  president. 

Initially,  many  community  college  administrators  were 
recruited  from  the  ranks  of  secondary  school  administrators 
(Sharp,  1984) .  A  proven  executive  ability,  rather  than  a 
distinguished  teaching  career,  became  an  increasingly 
important  qualification  for  appointment.  Those  candidates 
possessing  management  training  and  experience  or  advanced 
degrees  in  higher  education  became  available  in  increasing 
numbers.  By  the  mid-1970s,  the  selection  process,  responding 
to  equal  opportunity  laws  and  affirmative  action  commitments, 
acquired  a  public  character  and  produced  results  quite 
different  from  those  of  previous  generations.  Slowly,  the 
profile  of  the  college  administrator  changed  as  women  and 
minorities  took  office.  Sunshine  laws  further  complicated 
selection  in  several  states  and  appeared  to  some  observers  to 
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be  counterproductive  in  procuring  effective  leaders  (Sharp, 
1984)  . 

Achievement  in  these  administrative  positions  can  be 
influenced  by  complex  factors  beyond  a  candidate's  basic 
physical  and  mental  qualifications  for  the  job. 
Administrators  are  called  upon  to  carry  out  ministerial 
actions  which  require  skills  that  can  be  measured  empirically. 
But,  they  also  must  make  decisions  that  involve  judgment. 
Successful  administrative  performance  results  from  an 
interaction  of  at  least  ability,  personality,  motivation,  and 
situational  factors.  Consequently,  it  would  seem  that  these 
factors  should  be  included  in  the  selection  methods  used  to 
determine  managerial  talent   (Rawls  &  Rawls,   1974) . 

Researchers  have  tested  the  validity  of  personnel 
selection  methods  since  the  turn  of  the  century.  Munsterberg, 
a  Harvard  researcher  in  the  field  of  applied  psychology, 
developed  rudimentary  tests  for  the  selection  of  personnel  in 
various  manual  occupations.  His  research  laid  the  basis  for 
the  military  and  industrial  personnel  testing  done  during 
World  War  I  (Ghiselli,  1973).  A  post-war  surge  in  personnel 
selection  studies  focused  upon  the  identification  of  the 
essential  characteristics  people  needed  for  success  in 
specific  occupations  requiring  predominantly  physical  skills. 

After  World  War  II,  increasing  amounts  of  personnel 
selection  research  were  undertaken.    This  effort  was  put  forth 


mainly  by  private  corporations.  During  this  period,  selection 
methods  were  explored  that  would  help  employers  hire  people 
for  positions  that  required  administrative  skills.  The 
predominant  researchers  of  this  period,  including  Ghiselli 
(1955),  Halpin  (1954),  and  Hemphill  (1960),  found  that  it  was 
insufficient  to  attempt  to  match  a  person's  aptitudes  and 
personality  traits  to  a  generic  administrative  job 
description.  To  them,  defining  the  job  specifically  was  an 
important  part  of  finding  the  right  person  for  the  job.  Many 
researchers  agreed  that  one  single  test  or  examination  method 
probably  did  not  yield  sufficient  information  about  a 
candidate  to  ensure  a  proper  person-job  fit,  especially  for 
an  administrative  position. 

Determination  of  the  predictive  validity  of  a  test  or 
selection  method  is  fairly  easy  when  the  outcome  criterion, 
job  performance,  can  be  quantified.  The  presence  or  absence 
of  physical  and  some  mental  skills  is  easily  seen.  But  the 
ability  of  a  person  to  lead  others,  to  perform  well  under 
stress,  or  even  the  quality  of  a  person's  judgment,  often 
eludes  measurement. 

Managerial  performance  came  to  be  viewed  as  a  product  of 
many  interacting  variables  (Bray  &  Moses,  1972) .  Thus,  a 
movement  away  from  the  old  model  of  single  criterion  measures 
toward  a  systems  view  of  selection  was  undertaken.  As  a 
result,  comprehensive  systematic  models  which  took  into 
account    individual,    job,    and    organization    variables  were 


adopted.  In  so  doing,  the  selection  process  encompassed  the 
personal  characteristics,  training  needs,  and  organizational 
climate  factors  that  fostered  managerial  achievement. 
Adoption  of  this  more  complicated  view  involving  complex 
interactions  between  ability,  motivation,  and  opportunity 
variables  led  to  a  number  of  changes  that  affected  managerial 
selection  (Dunnette,  1963). 

One  of  these  changes  was  the  development  of  the  assessment 
center  approach.  During  the  mid-1950s,  American  Telephone  and 
Telegraph  (AT&T)  began  the  Management  Progress  Study.  For 
years  this  longitudinal  study  remained  the  most  valid  and 
scholarly  research  done  in  administrative  personnel  selection 
(Crooks,  197  3) .  The  assessment  center  concept  was  developed 
from  this  study  and  it  gained  rapid  acceptance  by  the  business 
community.  In  the  original  assessment  center  approach,  job 
candidates  were  subjected  to  multiple  tests  of  personality, 
ability,  and  motivation.  During  each  candidate's  three-day 
evaluation  he  was  subjected  to  the  traditional  tests  for 
knowledge  and  experience,  and  also  to  newer  methods  such  as 
job  simulations,  role-playing,  and  leaderless  group 
discussions.  Trained  observers  recorded  their  judgments  of 
each  candidate's  performance  during  the  entire  assessment 
process.  Hiring  personnel  hoped  to  select  the  most  suitable 
candidate  for  each  position  by  studying  the  results  of  the 
tests  and  the  observer's  judgments.  In  the  final  evaluation 
of    each    candidate    all    observer's    ratings    were  considered 
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collectively.  A  low  score  on  one  part  of  the  examination 
could  be  offset  or  disregarded  due  to  adequate  performance  on 
other  parts.  This  approach  replaced  the  previously  used 
"successive  hurdles"  method  in  which  a  failing  score  on  any 
one  part  of  an  selection  procedure  would  disqualify  a 
candidate  from  further  consideration  (Shoop,   1974) . 

By  the  1960s,  many  companies  and  public  administration 
organizations  had  implemented  the  assessment  center  approach. 
The  results  of  the  AT&T  study  were  so  convincing  that  many 
copied  all  or  part  of  the  original  process.  There  was  little 
additional  research  done  to  test  either  the  findings  of  the 
original  study  or  its  validity  in  a  new  situation  (Crooks, 
1973) .  During  this  time,  the  selection  of  administrators  in 
public  junior  and  community  colleges  has  continued  to  follow 
the  traditional  methods  of  reviewing  a  candidate's  education, 
experience,  references,  and  conducting  a  personal  interview. 
The  exercises  completed  in  hiring  college  and  university 
administrators  are  frequently  anything  but  a  process.  Too 
often  they  are  a  set  of  actions  which  leave  selection 
committees  dissatisfied  with  what  they  have  done  and 
candidates  dissatisfied  with  what  they  have  undergone.  No 
universal  process  exists  for  improving  what  often  is  an  inept 
or  discouraging  attempt  to  fill  an  administrative  vacancy  with 
the  best  available  person. 

The   problem    is   magnified   by   the    increasing   number  of 
searches  underway   in   institutions   of  higher  education.  In 
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many  colleges  and  universities  more  searches  are  underway  for 
administrators  than  for  faculty.  In  addition,  affirmative 
action  and  equal  opportunity  require  that  administrative 
searches  be  more  public,  more  extensive,  and  more  expensive 
than  ever  before.  All  this  expenditure  of  time  and  resources 
demands  that  more  attention  be  paid  to  the  selection  process 
(Kelly  &  Nelson,  1978). 

Much  has  been  written  about  the  problems  of  selecting 
administrators  in  higher  education,  however,  little  research 
has  been  done  on  the  topic.  A  change  in  the  procedures  now 
followed  may  be  warranted.  The  results  of  many  years  of 
research  in  personnel  selection  methods  in  other  fields  have 
been  usefully  applied  in  this  case.  The  conclusions  of  this 
research  helped  to  determine  guidelines  for  improving  the 
selection  of  administrators  for  our  public  junior  and 
community  colleges. 

Research  in  administrative  personnel  selection  methods  has 
been  done  in  corporations,  the  military  services,  public 
administrative  agencies,  and  in  education.  The  present  study 
was  needed  in  order  to  synthesize  this  research  and  apply  it 
to  the  selection  of  community  college  administrators.  The 
conclusions  of  the  research,  when  applied  to  this  problem, 
have  indicated  those  selection  methods  which  are  useful  in 
predicting  administrative  achievement. 
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Statement  of  the  Problem 
The    problem    addressed    in    this    study    was    to  develop 
guidelines  for  improving  the  selection  of  administrators  of 
public  junior  and  community  colleges. 

The  following  questions  were  addressed: 

1.  What  methods  for  the  selection  of  administrative 
personnel  are  commonly  used  by  corporations,  public 
administration  agencies,  the  military  services,  and 
institutions  of  higher  education? 

2.  What  methods  for  the  selection  of  administrative 
personnel,  derived  from  a  meta-analysis  of  selection  research, 
are  significantly  related  to  achievement  in  administrative 
positions? 

3.  What  information,  derived  from  a  meta-analysis  of  the 
methodologies  of  selection  research,  is  useful  in  interpreting 
the  findings  of  this  study? 

4.  What  guidelines  can  be  derived  from  this  research  for 
improving  personnel  selection  methods  for  administrators  of 
public  community  colleges? 

Justification  for  the  Study 
Radical  changes  have  occurred  in  the  public  workplace 
in  the  last  two  decades.  These  changes  have  required 
management  to  become  more  flexible  in  its  thinking  and  to 
reorient  itself  to  new  conditions.  In  higher  education  the 
advancements     in    tenure,     unionization,     equal  opportunity 
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legislation,    and   increased  protection   of   individual  rights 

present   community   college   administrators   with  more  complex 

situations  than  ever  before.     However,   selection  procedures 

for  these  positions  have   rarely  been  reviewed  to  determine 

their  continued  validity. 

Mclntyre  (1966)  stated  that  the  process  used  in  selecting 

higher  education  administrators  is  less  than  optimal. 

Of  all  the  rituals  encumbering  the  selection  process, 
interviewing  is  undoubtedly  the  hoariest — and  the 
sorriest.  Nothing  in  the  research  on  selection 
methodology  is  so  completely  established  and 
repeatedly  verified  as  is  the  unreliability  of  short 
interviews  as  they  are  usually  conducted. 
Unfortunately,  the  record  of  letters  of  recommendation 
is  as  dismal  as  that  of  interviewing.  Although  the 
subject  has  not  been  researched  to  any  great  extent, 
all  available  evidence  indicates  that  the  reading  of 
letters  of  recommendation  is  approximately  as 
enlightening  as  the  reading  of  tea  leaves.  Rating 
scales  vary  considerably  in  usefulness,  but  the  usual 
scale  is  little  if  any  better  than  the  usual  letter 
recommendation.  The  traits  to  be  rated  are  often  of 
limited  relevance,  the  points  on  the  scale  are  seldom 
clearly  defined,  and  leniency  is  so  rampant  that  only 
the  upper  end  of  the  scale  is  ordinarily  used.  (pp. 
7-8) 

In  seeking  to  improve  selection  methods,  every  phase  of 
the  process  should  be  investigated  and  reevaluated.  Much  can 
be  accomplished  when  those  in  control  of  an  organization  take 
a  broader  view  of  the  problem  of  selection  of  administrators 
and  allow  new  knowledge  to  influence  their  thinking. 
Occasionally  innovations  in  procedures  occur  serendipitously , 
but  more  often  they  are  the  result  of  long  hard  thinking, 
experimentation,     and    evaluation.        Purposeful    changes  in 
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organizational    methods    usually    grow    out    of    an  intensive 

investigation  of  data.     Sands   (1963)   stated  that 

further  study  is  especially  needed  in  the  area  of 
predicting  successful  managerial  functioning.  The 
forecast  for  success  can  be  improved  through 
correlation  studies  using  data  already  on  hand.  All 
that  is  necessary  is  the  willingness  to  search,  the 
ability  to  analyze,  and  the  imagination  to  interpret 
the  relationships  discovered.     (p.  188) 

The  conclusions  of  research  must  be  presented  in  a  usable 
form.  Two  decades  of  research  in  selection  of  administrative 
personnel  is  not  useful  to  those  selecting  administrators  of 
community  colleges  unless  it  is  analyzed  and  aimed  toward 
solving  that  specific  problem.  Silvern  (1971)  examined  15 
research  studies  by  education  systems  from  January  1965 
through  June  1967.  He  found  that,  although  the  published 
material  on  education  systems  had  been  proliferate, 
applications  of  this  material  were  found  in  only  1%  of  all  of 
the  literature.  The  present  study  attempts  to  fill  a  void  by 
presenting  the  results  of  24  years  of  administrative  selection 
research  in  a  usable  format. 

Delimitations 

The  delimitations  set  restrict  the  research  used  in  this 
study  to  a  specific  locale,  time  period,  phase  of  the 
selection  process,  and  type  of  position  considered. 

1.  The  research  reviewed  in  this  study  pertained  to  the 
following  areas:  (a)  corporations,  (b)  public  administration, 
(c)  military  services,   and   (d)  education. 
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2.  The  study  dealt  with  personnel  selection  methods  rather 
than  with  individual  selection  tests.  When  only  one 
particular  test  was  administered,  such  as  a  Strong  Vocational 
Interest  Blank,  this  test  was  recorded  by  the  type  of 
selection  method  it  represented. 

3.  The  methods  of  personnel  selection,  the  independent 
variable,  used  in  this  study  included  (a)  aptitude  and 
intelligence  measures,  (b)  personal  interviews,  (c)  job- 
related  skills  indicators,  (d)  psychological  attribute 
indicators,  (e)  value  systems  assessments,  (f)  biographical 
information,  (g)  peer  ratings,  (h)  self-appraisals,  and  (i) 
assessment  center  processes. 

4.  The  outcome  measures  of  administrative  achievement,  the 
dependent  variable,  used  in  this  study  included  (a) 
administrative  level  achieved,  (b)  salary  level  attained,  (c) 
supervisor  ratings,  (d)  number  of  years  serving  in  an 
administrative  position  within  the  same  organization,  (e) 
achievement  of  tenure,  (f)  objective  performance  data,  and  (g) 
promotion  rate. 

5.  Research  studies  which  defined  administrative 
achievement  as  measured  by  (a)  admissions  personnel  ratings, 
(b)  success  in  being  hired,  or  (c)  performance  success  in 
subseguent  training  programs,  were  not  used  in  this  study. 
These  criteria  were  judged  to  be  too  far  removed  from  any 
measurement  of  actual  performance  on  the  job. 
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6.  The  research  studies  analyzed  were  published  in  the 
United  States  from  January  1,  1962  through  December  31,  1985. 

Limitations 

The     following     confinements     were     observed     in  the 
investigation : 

1.  Performance  ratings  and  promotions  are  based  in  part 
upon  subjective  ratings  given  by  an  employee's  supervisor. 
As  such,  they  may  not  be  totally  objective  measures  of  job 
performance . 

2 .  A  lack  of  independence  between  the  predictor  and  the 
criteria  variables  may  occur  when  personnel  selection 
examination  results  are  made  available  to  an  employee's 
supervisor.  The  possible  effects  of  this  criterion 
contamination,  where  it  was  identified,  were  addressed  in  this 
study. 

3 .  Restriction  of  range  occurred  when  the  original 
population  or  sample  limited  itself,  or  was  limited  by, 
factors  which  related  to  the  selection  and  subsequent 
employment  process.  Thus,  if  50%  of  those  interviewed  were 
hired,  and  the  dependent  variable,  administrative  achievement, 
was  determined  only  for  those  hired,  the  effective  sample 
variance  for  the  study  was  cut  in  half.  This  affects  the 
subsequent  correlation  between  that  selection  method  and  any 
measure  of  administrative  achievement.     For  this  reason,  the 
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possible  effects  of  range  restriction,  where  they  were 
identified,  were  addressed  in  this  study. 

Definition  of  Terms 

Administrator .  An  administrator  is  any  person  who  manages 
or  directs  the  affairs  of  an  institution  or  any  major  part 
thereof.  As  pertains  to  higher  education,  anyone  above  the 
level  of  professor,  such  as  department  chairman  or 
professional  central  administrative  personnel,  is  considered 
an  administrator. 

Community  college.  A  community  college  is  a  two-year 
college,  offering  academic,  general  education,  vocational 
training,  terminal,  and  transfer  programs.  It  can  also  be 
known  as  a  junior  college. 

Corporation.  A  corporation  is  any  company  or  related 
group  of  companies  which  produces  goods  and/or  services  for 
profit.  Usually  this  type  of  organization  has  a  quantifiable 
measure  of  success  by  which  to  judge  the  effectiveness  of  the 
administrators  and  employees,  such  as  number  of  units  produced 
or  amount  of  profit  made.  Alternative  terms  for  "corporation" 
in  this  study  include  those  such  as  the  corporate  world, 
business  and  industry,   or  companies. 

Higher  education.  Higher  education  consists  of  public  and 
private  community  colleges,  four-year  colleges,  and 
universities  in  the  United  States. 
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Meta-analysis .  The  statistical  analysis  of  a  large 
collection  of  analysis  results  from  individual  studies  for  the 
purpose  of  integrating  the  findings. 

Military  services.  Military  services  are  national-level 
military  organizations  in  the  United  States. 

Public  administration.  Any  agency  which  provides  public 
services  at  the  federal,  state,  county,  city,  district,  or 
regional  level  is  considered  a  public  administrative  agency. 

Supervisor .  A  supervisor  is  a  person  who  oversees, 
directs,   or  manages  work  or  workers. 

Assumptions 

The  following  assumptions  have  been  made  in  this  study: 

1.  It  has  been  assumed  that  a  meta-analysis  of  the 
research  on  administrative  personnel  selection  was  an 
appropriate  methodology  for  determining  those  methods  which 
may,  when  properly  implemented,  improve  the  selection  of 
administrators  of  community  colleges. 

2 .  It  has  been  assumed  that  the  research  included  in  this 
study  was  guarded  against  restriction  of  range  and  criterion 
contamination  wherever  possible. 

Procedures 

The  purpose  of  this  study  was  to  develop  a  set  of 
guidelines  for  improving  the  selection  of  administrators  of 
public  community  colleges.      The  research  method  used  was  a 
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meta-analysis  of  administrative  personnel  selection  research 
done  in  the  United  States  in  the  past  24  years.  Procedures 
used  in  this  study  follow  those  described  by  Glass  (1977)  and 
in  Hunter,  Schmidt,  and  Jackson  (1982)  for  the  meta-analysis 
of  a  sample  of  independent  correlational  studies.  The  steps 
involved  included  the  following: 

1.  Identification  of  the  research  studies. 

2.  Classification  of  each  study  dependent  variable, 
administrative  achievement,  according  to  the  Delimitations 
section  of  this  chapter. 

3.  Classification  of  each  study  independent  variable, 
selection  method,  according  to  the  Delimitations  section  of 
this  chapter. 

4.  Calculation  of  frequencies  of  selection  methods  used 
to  answer  the  questions:     (a)  What  were  the  most  commonly  used 
selection  methods  across  all  types  of  organizations,   and  (b) 
what  were  the  most  commonly  used  selection  methods  in  each 
type  of  organization. 

5.  Calculations  of  the  means  and  standard  deviations  from 
the  means  in  selected  cross-sections  of  the  data  were  used  to 
determine  which  selection  methods  were  significantly  related 
to  achievement  in  administrative  positions. 

6.  Determination  of  what  information,  derived  from  the 
meta-analysis  of  the  methodologies  of  the  research  studies 
used,  can  be  useful  in  interpreting  the  findings  of  the  above 
analysis  of  selection  methods. 
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7.  Analyzation  of  the  findings  of  the  above  questions  to 
determine  how  the  frequency  of  use  of  each  selection  method 
related  to  its  ability  to  predict  administrative  achievement. 

8.  Derivation  of  guidelines  useful  in  improving  personnel 
selection  methods  for  administrators  of  public  community 
colleges . 

9.  Presention  of  the  data  in  textual  form  supported  by 
frequency  tables  and  figures,  and  supported  by  tables  and 
figures  reporting  means  and  standard  deviations  of  the 
correlational  data. 

Organization  of  the  Dissertation 
A  review  of  the  literature  pertaining  to  administrative 
personnel  selection  methods  is  provided  in  Chapter  Two. 
Chapter  Three  presents  a  detailed  discussion  of  the  meta- 
analytic  research  procedures  used  in  this  study,  such  as 
methodology,  sources  of  data,  data  analysis  techniques,  and 
presentation  of  the  data.  A  presentation  of  the  findings  of 
the  meta-analysis  of  the  research  studies  used  is  contained 
in  Chapter  Four,  thus  answering  the  questions  posed  in  the 
Statement  of  the  Problem.  Chapter  Five  contains  the 
guidelines  derived  from  the  meta-analysis  of  the  research 
studies  and  the  presentation  of  these  guidelines  for 
application  to  the  selection  of  community  college 
administrators . 


CHAPTER  II 
BACKGROUND  LITERATURE 

Introduction 

The  responsibilities  inherent  in  administrative  positions 
have  undergone  a  complex  evolution  since  the  turn  of  the 
century.  In  many  organizations,  however,  the  selection 
process  for  administrative  personnel  has  not  seen  appreciable 
change.  In  this  review,  the  focus  is  on  the  research 
literature  pertaining  to  the  usefulness  of  various  selection 
procedures  in  the  prediction  of  administrative  success.  The 
review  is  organized  into  the  following  sections:  the  history 
of  personnel  selection;  a  discussion  of  objectivity, 
reliability,  and  validity  in  personnel  selection  research;  a 
review  of  the  personnel  selection  methods  used  as  independent 
variables  in  personnel  selection;  introduction  of  the  research 
studies  reviewed;  and  a  summary  and  critique. 

History  of  Personnel  Selection 

Traditionally,  Munsterberg ' s  1911  experiment  with  motormen 
has  been  viewed  as  the  beginning  of  research  in  the  use  of 
examinations  for  personnel  selection  (Ghiselli,  1973)  . 
Nevertheless,  anecdotal  evidence  leads  one  to  suggest  that, 
even  before  1910,  other  psychologists  conducted  similar 
studies  with  tests,   but  these  were  small   in  scope  and  went 
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unpublished.  Under  the  impetus  of  the  scientific  management 
movement  of  the  early  1900s,  some  efficiency  experts  of  that 
time  were  using  simple  exams  for  evaluating  applicants  for 
jobs.  They  reported  fragmentary  evidence  of  validity  in  the 
attempt  to  justify  their  activities  (Ghiselli,  1973).  The 
earliest  review  of  an  industrial  application  was  in  1915  when 
Scott  (in  Arvey  &  Campion,  1982,  p.  283)  reported  low 
reliability  for  evaluations  given  by  six  personnel  managers 
who  had  interviewed  the  same  36  sales  applicants. 

During  World  War  I,  the  large-scale  testing  both  of 
soldiers  and  industrial  workers  provided  stimulation, 
methodology,  and  respectability  to  the  examination  of  the 
utility  of  examinations  in  the  assessment  of  occupational 
aptitude.  In  1923,  Freyd  (in  Guion,  1987)  published  a  10-step 
outline  for  personnel  selection.  This  outline  was  so  complete 
that  it  differs  little  from  the  procedures  used  in  personnel 
selection  research  today.  This  all  led  to  a  post-war  surge 
of  systematic  research  in  personnel  testing. 
Selection  Processes  in  Business  and  Industry 

Prior  to  World  War  I,  most  businesses  were  owner-managed 
by  the  individuals  who  had  founded  them.  But  the  growth  of 
the  economy  since  the  1920s  brought  changes  which  have 
replaced  these  colorful  characters  with  impersonal 
corporations.  The  function  of  the  administrator  had  become 
such  that  it  reguired  many  skills  and  much  knowledge,  so  that 
it  often  took  a  number  of  administrative  personnel  to  assist 
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in  decision  making.  By  the  1960s,  corporate  management 
required  a  knowledge  of  a  variety  of  fields  and  operations 
rarely  needed  in  business  a  few  years  earlier.  Corporations 
were  growing,  combining,  and  expanding  so  fast  that  they 
needed  to  hire  competent  executives  at  a  pace  which  could  not 
be  fulfilled  by  promotion  from  within  the  ranks  (Sands,  1963, 
p.   3)  . 

One  particular  study  was  more  comprehensive  than  much  of 
the  personnel  selection  research  of  the  1950s  and  1960s.  This 
was  the  American  Telephone  and  Telegraph  (AT&T)  (Bray  &  Grant, 
1966)  study  from  which  came  the  assessment  center  method  for 
the  selection  of  administrative  personnel.  Called  the 
Management  Progress  Study,  initiated  in  1956,  it  is  the  first 
known  corporate  research  and  application  of  the  assessment 
center  method.  The  422  men  brought  into  the  study  were 
followed  up  annually  and  reassessed  at  7-year  intervals  in  a 
effort  to  keep  track  of  their  professional  development.  The 
assessment  center  results  were  not  revealed  to  the  higher  AT&T 
management  personnel,  so  that  the  progress  of  the  men  would 
not  be  affected  by  the  assessment  findings.  The  results  of 
this  study  have  been  well  documented.  Between  1961  and  1967, 
only  this  AT&T  study  and  two  studies  done  by  the  armed  forces 
on  assessment  centers  were  reported  in  the  literature 
(Crooks,    1973,   p.  1). 
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Selection  Processes  in  Public  Administration 

When  public  administration  systems  were  initiated,  it  was 
assumed  that  only  people  of  the  highest  caliber  would  apply 
for  service.  Once  reality  dashed  this  assumption,  public 
administrative  personnel  selection  systems  were  built  upon  the 
cornerstone  of  the  competitive  examination,  a  method  of 
deselecting  inferior  job  candidates  (Stahl,  1976,  p.  129). 
From  the  beginning,  this  system  has  placed  special  emphasis 
upon  formal  selection  procedures  such  as  testing  for  specific 
knowledge  and  skills  related  to  the  particular  job  the  person 
was  applying  for.  The  starting  points  for  the  selection 
process  are  basically  two:  a  determination  of  the  objective 
for  selection,  which  may  be  a  given  position,  occupation, 
program,  or  service-wide  career,  and  a  setting  of  basic 
standards  for  selection,  the  skills  and  knowledge  that  are 
necessary  to  meet  the  preceding  objective.  During  the  early- 
to  mid-1960s,  some  public  administration  organizations 
realized  a  need  to  ensure  at  least  an  adequate  intake  of  high 
caliber  people  at  all  levels  so  that  there  would  be  no 
shortage  of  talent  when  movement  upward  or  outward  took  place. 
Thus,  some  services  began  to  give  more  attention  to  the 
selection  of  persons  who  possessed  a  capability  of  growth  and 
development.  To  this  end,  public  administrative  organizations 
began  to  implement  the  assessment  center  approach. 

However,  most  public  administrative  organizations  select 
a  person  to   fill   a  particular  job  vacancy,    and  thus  their 
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testing  is  for  a  specific  competency  in  that  area,  not  an 
overall  assessment  of  the  applicant's  potential.  Most  of 
these  agencies  expect  personnel  to  have  learned  their  skills 
previous  to  application  and  do  not  intend  to  take  in  an 
inexperienced  person  of  promise  and  then  train  him  or  her  for 
an  administrative  position. 

Most  of  the  background  studies  cited  as  a  basis  for 
personnel  selection  in  public  administration  were  actually 
conducted  by  private  corporations.  Many  of  the  selection 
practices  have  been  borrowed  and  adapted  from  business 
research.  AT&T's  original  industrial  development  of  the 
assessment  center  method  was  so  well  grounded  in  research,  it 
was  often  casually  implied  that  therefore  any  assessment 
center  would  pick  the  "right"  person  (Ross,  1979,  p.  41). 
Selection  Processes  in  the  Military  Services 

Although  many  fields  acguire  administrators  already 
trained  for  the  positions  they  are  hired  to  fill,  the  military 
services  often  must  select  people  of  officer  guality  and  then 
pay  the  additional  cost  of  training  them.  Thus,  the  basic 
objective  in  the  selection  process  is  to  identify  measures 
which  would  result  in  officers  entering  the  force  with  a  high 
probability  of  success.  The  services  must  select  personnel 
not  only  to  fill  a  particular  position  as  it  becomes 
available,  but  also  to  identify  people  who  have  the  ability 
to  be  trained  repeatedly,   sometimes  every  three  years,   for  a 
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variety  of  different  positions  (Akman  &  Nordhauser,  1974,  p. 
3)  • 

Often,  the  services  must  select  officer-quality  personnel 
to  fill  positions  which  have  no  equivalent  in  the  nonmilitary 
world.  In  these  cases,  either  very  global  criteria  must  be 
set  to  determine  if  the  candidate  has  the  intelligence  or 
aptitude  to  learn  the  job-required  skills  and  then  be 
successful  at  it,  or  very  specific  abilities  or  natural 
characteristics  must  be  measured  to  ensure  the  candidate  will 
be  well  suited  for  an  important  aspect  of  the  position. 
Because  a  large  number  of  people  must  be  tested  and  screened 
for  placement,  the  military  services  have  developed  many 
paper-and-pencil  tests  to  measure  both  general  and  specific 
aptitudes.  They  have  relied  heavily  upon  past  achievement, 
such  as  college  degrees  and  the  results  of  officer  candidacy 
tests.  The  importance  of  test  scores  and  military  class 
standings  is  shown  by  the  fact  that  often  one's  scores  and 
future  attainment  of  rank  are  positively  correlated. 

Selection  methods  used  in  the  military  services  and  in 
public  administration  are  often  classified  as  "successive 
hurdles"  techniques.  A  lower  than  acceptable  score  on  any  one 
particular  screening  test,  or  part  of  a  test,  will  either 
disqualify  the  candidate  from  that  service  or  from  particular 
jobs  within  the  service   (Shoop,   1974,  p.  341). 
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Selection  Processes  in  Higher  Education 

In  higher  education,  search  committees  are  often  used  for 
recruiting  administrative  personnel.  These  committees  provide 
for  maximum  participation  in  the  selection  process  by  a 
variety  of  constituencies  within  the  institution.  The 
membership  of  the  search  committee  depends  upon  the  vacancy 
to  be  filled.  The  committee  may  include  faculty  and  staff 
members,  administrators,  and,  in  some  cases,  students.  A 
member  of  the  central  personnel  staff  is  often  an  ex-officio 
committee  member  in  order  to  orient  the  committee  to  the 
proper  and  legal  selection  procedures  (Sprunger  &  Bergguist, 
1978,  p.  116).  Usually,  the  search  committee  recruits, 
screens  candidates,  checks  references,  participates  in 
preliminary  interviewing,  and  recommends  a  fixed  number  of 
candidates  to  a  designated  administrator,  who  makes  the  final 
selection   (Fortunato  &  Waddell,   1981,  p.  107). 

The  initial  screening  of  candidates  has  usually  been  done 
by  a  review  of  the  vitae  and  job  applications.  Supplemental 
information  has  usually  been  obtained  by  talking  to  references 
and  others  who  knew  the  individuals  and  their  work.  The  on- 
campus  interview  has  been  one  of  the  most  important  steps  in 
the  selection  process.  The  manner  in  which  this  interview  is 
conducted  has  usually  been  critical  to  the  success  of  the 
recruitment-selection  process  (Sprunger  &  Berguist,  1978,  p. 
119).  Webster  (cited  in  Grove,  1981,  p.  56)  emphasized  that 
the  interviewer  must  understand  what  behavior  is  reguired  of 
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the  person  finally  selected  to  fill  the  position.    He  stressed 

such  practices   as   reviewing  the   application   form  with  the 

applicant,  giving  the  applicant  an  opportunity  to  talk  about 

himself  or  herself   in  a  way  that  helps  that   individual  to 

present     relevant     evidence,     making     the     interview  more 

systematic    and    probing,     examining    test    scores    and  other 

information,   and  weighting  the  various  types  of  information 

in     order     to     reach    an     intelligent     decision.  Webster 

recommended  taking  time  following  each  interview  to  clarify 

impressions  and  to  formulate  a  judgment.    He  suggested  making 

use  of  multiple  independent  evaluations  in  which  two  or  more 

interviewers  record  decisions  independently  and  subsequently 

reconcile  differences. 

Objectivity.  Reliability,  and  Validity  in  Personnel 

Selection  Research 

These  three  primary  considerations  enter  into  the 
effectiveness  and  propriety  of  examination  processes  for 
employment  selection:  objectivity,  reliability,  and  validity. 
Objectivity  is  even  more  important  in  the  1980s  than  it  was 
previously  due  to  the  legal  demand  for  testing  to  have  a  job- 
specific  relation.  Reliability  is  important  but  is  often  not 
reported  in  personnel  selection  research.  Validity  is  the 
major  concern  of  this  dissertation,  and  the  research  studies 
reviewed  here  pertain  to  the  validity  of  each  selection  method 
in  predicting  administrative  achievement. 


25 

Objectivity 

According  to  Stahl  (1976),  one  of  the  prime  reasons  for 
professionalizing  all  steps  in  the  selection  process  is  to 
ensure  thoroughgoing  objectivity  (p.  131) .  Only  those  methods 
which  disregard  extraneous  factors  such  as  race,  religion, 
politics,  sex,  residence,  and  age  can  be  considered  thoroughly 
objective.  An  objective  selection  method  should  identify 
those  characteristics  of  mind  and  skill  necessary,  and  only 
those  necessary,  to  the  given  purpose,  whether  the  purpose  is 
to  fill  a  particular  position  or  to  begin  a  career. 

Objectivity  is  not  only  desirable  but  is  mandated  in 
hiring  for  positions  in  the  public  sector.  Throughout  almost 
all  of  American  history,  legislation  has  been  enacted  to 
facilitate  egual  treatment  in  employment.  An  abundance  of 
case  law  has  built  up  around  civil  rights  legislation,  and 
rulings  have  occurred  which  have  broadened  the  impact  of  such 
legislation.  This  broadening  has  had  the  effect  of  placing 
more  and  more  constraints  on  what  employers  are  and  are  not 
allowed  to  do  in  terms  of  employee  selection  (Burrington, 
1982,   p.   55) . 

Two  landmark  cases  have  set  the  serious  tone  with  which 
violations  of  civil  rights  in  hiring  will  be  met.  In  the  U.S. 
Supreme  Court  case  of  Griggs  vs.  Duke  Power  (401  U.S.  424, 
1971) ,  an  employer  rejected  black  job  applicants  on  the  basis 
of  lack  of  completion  of  high  school  or  on  the  results  of  a 
general  intelligence  test.    Evidence  showed  that  employees  who 
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had  not  completed  high  school  or  taken  the  tests  had  continued 
to  perform  satisfactorily  and  made  progress  in  departments  for 
which  the  high  school  and  test  criteria  were  not  used 
(Burrington,  1982) .  It  was  ruled  that  employment  standards 
and  tests  which  are  not  significantly  related  to  job 
performance  violate  the  Civil  Rights  Act  of  1964,  Title  VII 
(Alexander,   1980,   p.   515) . 

The  second  case,  Edward  L.  Kirkland  et  al..  Plaintiffs  v. 
New  York  State  Department  of  Correctional  Services  et  al.. 
Defendants  (73  LIV.  1548,  1974),  was  brought  under  the  Fifth 
and  Fourteenth  Amendments  to  the  U.S.  Constitution  and  under 
the  Civil  Rights  Acts  of  1866  and  1871.  The  case  concerned 
a  Correction  Sergeant  (male)  written  examination  which  had  the 
impact  of  allowing  only  1.9%  of  blacks,  and  no  Hispanics,  to 
be  eligible  for  promotion.  No  recourse  was  made  to  the  Civil 
Rights  Act  of  1964,  so  the  Equal  Employment  Opportunity 
Commission  (EEOC)  Guidelines,  required  under  that  act,  were 
not  binding.  However,  the  U.S.  District  Court,  Southern 
District,  New  York,  considered  these  Guidelines  in  evaluating 
the  job  relatedness  of  the  examination.  A  number  of 
implications  can  be  deduced  from  this  case  for  personnel 
selection  in  general  and  the  examination  process  in  particular 
(Wisner,    1975,   pp.   266-267) . 

1.  In  rendering  a  decision,  the  court  is  likely  to 
follow  the  EEOC  Guidelines  even  though  the  case  may  not  have 
been  brought  under  the  Guidelines. 
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2.  A  job  analysis  should  be  performed  prior  to  the 
development  of  each  examination. 

3.  Those  critical  elements,  or  more  important 
characteristics  of  the  job,  which  distinguish  between  those 
likely  to  be  successful  and  those  less  likely  to  be 
successful,  must  be  included  in  the  examination. 

4.  Subject  matter  experts,  to  be  considered  qualified, 
must  be  of  an  appropriate  rank  and  sufficiently  experienced 
in  the  current  skills,  knowledges,  and  abilities  required  in 
the  job  for  which  the  test  is  developed. 

5.  Discriminatory  test  results  cannot  be  refuted  by 
demonstrating  that  the  results  occurred  because  of  certain 
environmental  conditions  or  cultural  characteristics  related 
to  the  protected  group  population  unless  it  can  be  shown  that 
the  examination  is  job-related. 

6.  The  fact  that  specific  parts  of  the  test  are  not 
found  to  be  discriminatory  does  not  serve  to  support  the  use 
of  an  examination  whose  overall  results  are  found  to  be 
discriminatory . 

7.  All  subtests  included  in  the  examination  must  be 
weighted  in  accord  with  their  importance  to  the  job. 

8.  Data  should  be  obtained  concerning  the  examination 
results  immediately  after  administering  the  exam  to  determine 
the  examination's  impact  on  protected  groups. 

9.  Every  effort  should  be  made  to  set  passing  scores  at 
a  point  considered  to  separate  the  candidates  most  likely  to 
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be  successful  from  those  considered  to  have  the  least  chance 
of  success. 

10.  Be  prepared  to  answer  the  question  "why  was  this 
method  of  measurement  considered  the  best  means  to  employ?" 

11.  In  court,  do  not  rely  too  heavily  on  your 
professional  witness  to  attest  to  the  job  relatedness  of  your 
examination. 

Thus,  the  public  employer  has  but  one  choice — to  validate 
the  employee  selection  procedures.      Fortunately,   the  choice 
is    the    best    one    to    follow    from    the    standpoint    of  good 
personnel  management. 
Reliability 

The  consistency  with  which  a  method  measures  what  it  is 
expected  to  measure  is  called  reliability  (Ary,  Jacobs,  & 
Razavieh,  1979,  p.  206).  In  order  to  achieve  consistency,  a 
selection  test  or  method  must  leave  little  room  for  chance  in 
the  subject's  final  score  or  rating.  If  a  test  is  reliable, 
a  person  taking  it  at  two  different  times  should  make 
substantially  the  same  score  or  be  ranked  in  approximately  the 
same  position  each  time.  Test  reliability  depends  also  on  the 
nature  of  the  variable  being  measured.  For  example,  the  trait 
of  academic  achievement  can  be  measured  more  consistently  than 
that  of  personality.  Although  all  measurements  of  human 
qualities  contain  some  error,  no  test  or  method  of  personnel 
selection  is  of  value  unless  it  has  a  high  degree  of 
reliability  (Stahl,   1976,  p.  133). 
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Reliability  can  be  affected  by  varied  administration  of 
a  selection  procedure  and  by  the  scoring  or  judging  of  an 
applicant's  performance.  Inexperienced  persons  or  committees 
may  deviate  from  standardized  or  prescribed  procedures  in 
testing  an  applicant.  Vague  observational  techniques  or 
scoring  of  an  applicant's  behavior  or  responses  leads  to  an 
unreliability  which  has  nothing  to  do  with  the  competency  of 
the  candidate.  The  difficulty  of  any  selection  method  or 
examination  affects  its  reliability.  When  a  method  is 
difficult,  the  subjects  guess  on  most  of  the  questions  and  a 
low  reliability  results.  Conversely,  if  the  type  of 
examination  is  easy,  all  subjects  have  correct  responses  on 
most  of  the  items  and  only  a  few  more  difficult  items 
discriminate  among  subjects.  This  again  results  in  a  low 
reliability.  Thus,  any  selection  committee  must  keep  in  mind 
the  fact  that  they  must  adjust  the  substance  of  questions 
asked  in  order  to  discriminate  subtle  differences  between 
similar  candidates  on  traits  that  are  difficult  to  measure. 

The  achievement  of  a  high  degree  of  reliability  in 
selection  procedures  for  administrative  positions  in  a 
community  college  may  be  difficult.  In  addition  to  the 
problems  inherent  in  assessing  traits  which  elude  measurement 
by  persons  who  are  often  inexperienced  in  the  task, 
reliability  can  be  affected  by  the  lack  of  heterogeneity  of 
the  group  of  applicants.  The  greater  the  homogeneity  of  the 
group  of  subjects  being  considered,  the  lower  the  reliability 
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of  any  selection  method  in  differentiating  among  those 
subjects.  Candidates  for  these  administrative  positions  are 
likely  to  be  guite  similar  in  the  traits  being  measured. 
Therefore,  it  is  more  difficult  to  rank  them  than  it  would  be 
to  rank  the  same  number  of  subjects  chosen  randomly  from  the 
entire  population.  This  restriction  in  range  is  a  problem  in 
most  personnel  selection  research  as  very  seldom  is  the  sample 
drawn  from  the  population  at  large. 
Validity 

Validity  is  defined  as  the  extent  to  which  an  instrument 
or  method  measures  what  it  is  intended  to  measure.  Unlike  the 
physical  sciences,  where  there  is  a  direct  means  of  measuring 
the  outcome  variable,  in  personnel  selection  indirect  means 
must  be  used  to  measure  complex  attributes. 

The  type  of  validity  addressed  in  this  study  is  criterion- 
related  validity.  Criterion-related  validity  refers  to  the 
relationship  between  the  scores  on  a  measuring  instrument  and 
an  independent  external  variable  (criterion)  believed  to 
measure  directly  the  behavior  or  characteristic  in  guestion 
(Ary,  Jacobs,  &  Razavieh,  1979)  .  In  this  type  of  validity  the 
emphasis  is  on  the  criterion  rather  than  on  the  instrument 
itself.  One  is  primarily  interested  in  what  the  instrument 
can  predict  rather  than  in  the  test  content. 

There  are  several  characteristics  that  a  criterion  measure 
should  possess.  One  must  judge  whether  the  criterion  chosen 
as    the    dependent    variable    really    represents  successful 
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performance  of  the  behavior  in  question.  If  the  criterion 
does  not  reflect  the  attribute  under  study,  it  would  be 
meaningless  to  use  it  as  a  basis  for  validating  another 
instrument.  In  the  case  of  this  study,  the  outcome  criteria 
of  supervisor's  ratings,  longevity  in  a  position,  and 
salary/position  or  rank  attained  were  deemed  to  be  adequate 
indicators  of  success  in  an  administrative  position. 

A  second  characteristic  is  that  a  criterion  must  be 
reliable.  The  criterion  must  be  a  consistent  measure  of  the 
attribute  over  time  or  from  situation  to  situation.  The 
criteria  chosen  for  this  study,  rank,  position,  and  salary 
achieved,  have  generally  been  considered  to  be  related  to 
success.  Length  of  time  in  an  administrative  position  within 
one  organization  may  have  been  a  mark  of  high  achievement  in 
the  past  more  than  it  is  today.  Now  many  administrators  look 
upon  great  longevity  as  an  indicator  of  stagnation  rather  than 
of  achievement.  So  this  indicator  may  not  be  as  relevant  to 
later  studies  as  to  earlier  ones. 

Finally,  a  valid  criterion  should  be  free  from  bias.  The 
scoring  should  not  be  influenced  by  any  factors  other  than 
actual  performance.  The  main  problem  in  relation  to  personnel 
selection  research  is  a  form  of  bias  called  criterion 
contamination.  This  occurs  when  an  individual's  score  on  the 
criterion,  such  as  supervisor  rating,  is  influenced  by  the 
scorer's  knowledge  of  the  subject's  predictor  score. 
Contamination    of    the    criterion    can    be    prevented    by  not 
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permitting  the  person  who  grades  or  rates  the  criterion  to  see 
the  predictor  scores. 

Although  perfect  criterion-related  predictive  validity  is 
practically  impossible  to  achieve,  a  reasonable  amount  of 
validity  is  possible.  Selection  upon  the  basis  of  tests  or 
methods  which  have  no  known  validity  may  be  little  different 
from  selection  upon  the  basis  of  a  turn  of  a  card.  Yet  some 
personnel  agencies  have  chosen  to  use  methods  which  have  not 
been  subjected  to  validity  study. 

The  process  of  determining  the  validity  of  a  selection 
method  involves  statistical  correlation  between  the  various 
examination  results  and  some  criteria  of  performance  on  the 
job.  Where  the  worker  is  engaged  in  production  activities  in 
which  it  is  easy  to  measure  output,  the  question  of  the 
criterion  presents  no  problem.  But  the  simplicity  of  the 
problem  disappears  when  one  tries  to  find  adequate  criteria 
of  performance  for  an  administrator.  In  these  cases,  the 
researcher  must  rely  upon  appraisals  by  those  closely 
acquainted  with  the  work  of  the  particular  administrator 
(Stahl,   1976,   p.    132) . 

Kirchner  and  Reisberg  (1962)  pointed  out  the  problem  of 
determining  an  adequate  criteria  of  managerial  performance  (p. 
301).  Supervisors'      ratings      of      the      performance  of 

administrative  personnel  were  found  to  be  subjective  and 
widely  varied.  Successful  supervisors  preferred  subordinate 
administrators   who   showed    initiative   toward  organizational 
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goals,  whereas  supervisors  considered  to  be  less  successful 
rated  their  subordinate  administrators  more  highly  who 
followed  orders  and  got  along  with  others. 

There  are  two  main  procedures  by  which  the  validity  of  an 
examination  or  method  for  selection  is  best  determined.  Each 
is  supplementary  to  the  other.  In  the  first  instance, 
concurrent  validity,  the  exam  may  be  given  to  administrators 
of  known  ability  already  on  the  job.  If  those  administrators 
who  have  been  predetermined  to  be  most  successful  score 
highest  in  the  evaluation,  while  the  least  successful  ones 
score  the  lowest,  the  evaluation  method  is  said  to  have 
evidence  of  validity.  The  second  procedure  consists  of  long- 
term  follow-up  studies  of  the  performance  of  those  employees 
who  have  been  selected  for  administrative  positions  (Stahl, 
1976,   p.    132) . 

Review  of  the  Research  on  Selection  Methods 
for  Administrative  Personnel 

Methods  for  personnel  evaluation  have  been  described  in 

as   many   different   ways   as   there   have   been   researchers  to 

describe  them.      According  to  Northcott   (1960,   p.    289),  the 

procedures   in   standard  use  have   fallen   into   three  groups: 

questionnaires,  tests,  and  interviews.     Stahl   (1976,  p.  136) 

stated    that    the    forms    which    examination    can    take    may  be 

classified  into  five  categories:     (a)  a  systematic  evaluation 

of  education  and  experience,    (b)  oral  tests,    (c)  standardized 

qualification  inquiries,    (d)  written  tests,  and  (e)  tests  of 


personality  traits.  According  to  the  "Uniform  Guidelines  on 
Employee  Selection  Procedures,"  issued  by  the  Equal  Employment 
Opportunity  Commission,  the  United  States  Department  of 
Justice,  the  United  States  Civil  Service  Commission,  and  the 
United  States  Department  of  Labor  in  1978,  selection 
procedures  are  defined  as  including  "the  full  range  of 
assessment  techniques  from  traditional  paper  and  pencil  tests, 
performance  tests,  training  programs,  or  probationary  periods 
and  physical,  educational,  and  work  experience  requirements 
through  informal  or  casual  interviews  and  unscored  application 
forms"   (Quaintance,   1980,   p.  126). 

The  selection  methods  reviewed  in  this  study  have  been 
delineated  into  the  following  types:  (a)  aptitude  and 
intelligence  measures,  (b)  personal  interviews,  (c)  job- 
related  skills  indicators,  (d)  psychological  attribute 
indicators,  (e)  value  systems  assessments,  (f)  biographical 
information,  (g)  peer  ratings,  (h)  self  appraisals,  and  (i) 
assessment  center  processes.  These  methods  are  the 
independent,  or  predictor  variables  of  the  administrative 
personnel  selection  research  studies  used  in  this  meta- 
analysis. Statistical  data  and  methodological  details  of  each 
of  the  research  studies  reviewed  for  this  dissertation  are 
found  listed  alphabetically  by  researcher's  name(s)  in  the 
Appendix. 
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Aptitude  and  Intelligence  Measures  as  a  Selection  Method 
In  a  large  proportion  of  the  research,  at  least  one  type 
of  intelligence  or  mental  aptitude  test  has  been  used.  Most 
of  the  tests  utilized  measured  verbal  abilities,  but  some 
measured  critical  thinking,  or  mathematical  or  mechanical 
aptitudes . 

In  a  review  of  12  research  studies  aimed  at  relating  a 
measure  of  aptitude  or  I.Q.  to  later  measures  of 
administrative  achievement,  Korman  (19  68)  found  that  the 
"typical  managerial  applicant  population  is  already  highly 
preselected  on  abilities  and  is  relatively  homogeneous  on 
these  variables"  (p.  301) .  Thus,  the  tests  of  ability  do  not 
tend  to  discriminate  finely  among  such  a  population  in  which 
preselection  has  already  taken  place.  Korman  states  that 
"verbal  ability  tests  seem  to  show  little  usefulness  for 
predicting  managerial  performance  above  the  first-line 
supervisory  level"   (p.   297) . 

Dating  back  into  the  1940s,  cognitive  ability  research 
showed  mixed  results.  Williams  and  Leavitt  (1947) ,  using 
Marine  Corps  officers  as  subjects,  showed  no  significant 
relationship  with  supervisor  ratings.  However,  Knauft  (1949) 
found  a  significant  relationship  with  later  ratings.  In  the 
1950's  Handyside  and  Duncan  (1954)  and  Meyer  (1956),  each 
using  supervisors  as  subjects,  found  significant  relationships 
with  ratings,  but  Comrey  and  High  (1955) ,  Ricciuti  (1955) ,  and 
Thorndike  and  Hagen  (1959)   did  not. 
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Research  used  in  this  study.  A  total  of  15  research 
studies  were  found  which  used  aptitude  and  intelligence 
measures  as  a  predictor  variable  and  met  the  other 
delimitations  of  this  dissertation.  These  studies  yielded  21 
correlation  coefficients  which  ranged  in  strength  from  r  =  .01 
to  r  =  .41.     Samples  ranged  in  size  from  26  to  8,885  persons. 

Hicks  and  Stone  (1962) ,  while  working  for  Aerojet-General 
Corporation  and  California  State  Polytechnic  College, 
respectively,  evaluated  the  effectiveness  of  a  test  battery 
in  discriminating  between  successful  and  unsuccessful 
managers.  One  part  of  this  concurrent  study  related  the 
mental  ability  of  managers,  supervisors,  shop  foremen,  and 
engineering  supervisors,  to  the  outcome  measure,  supervisor 
rating.  Results  were  not  significant.  The  researchers 
concluded  that  mental  abilities  did  not  show  a  large 
relationship  to  managerial  success  probably  as  the  result  of 
sample  attenuation  through  preselection  on  education  and 
career  achievement. 

Williams  and  Harrell  (1964),  while  at  San  Francisco  State 
College  and  Stanford  University,  respectively,  did  a  follow- 
up  study  of  Stanford  MBA  graduates.  The  purpose  was  to  find 
predictors  which  were  significantly  correlated  with  on-the- 
job  achievement.  While  the  grade  point  averages  for 
undergraduate  courses  and  for  reguired  graduate  courses  fell 
short  of  significant  correlations  with  the  success  criterion, 
there  was  a  significant  correlation  between  grades  on  elective 


graduate  courses  and  salary.  The  researchers  suggested  that 
research  attention  might  be  directed  at  some  figure  which 
would  concentrate  on  the  elective  course  area. 

Dicken  and  Black  (1965) ,  while  at  Stanford  University, 
explored  the  validity  of  clinical  interpretations  of  an 
objective  test  battery  in  two  different  corporate  settings. 
Subjects  were  first-level  supervisors  in  a  manufacturing 
organization  and  in  an  insurance  company.  Only  the  highest 
of  the  derived  correlations  was  significant.  The  researchers 
acknowledged  the  restricted  range  of  their  samples  and  its 
effect  on  predictability,  stating  that  "measures  of  ability 
and  interest  cannot  be  expected  to  make  fine  discriminations" 
(p.  46). 

In  1965,  Tenopyr  and  Ruch  (in  Campbell,  Dunnette,  Lawler, 
&  Weick,  1970)  studied  the  relationship  between  intelligence 
test  results  and  the  salary  level  attained  by  corporate 
production  managers.  The  resulting  correlation  coefficient 
in  this  concurrent  study  was  significant. 

Bray  and  Grant  (1966) ,  while  with  American  Telephone  and 
Telegraph  Company,  sought  to  study  the  assessment  center 
process.  This  study  has  become  the  landmark  study,  to  date, 
in  assessment  center  development.  A  sample  of  low-level 
management  employees  of  AT&T  were  tested  in  this  carefully 
designed  assessment  center.  This  was  one  of  the  few  studies 
in  personnel  selection  research  where  no  criterion 
contamination  was  allowed.    A  follow-up  of  the  subjects  eight 
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years  after  testing  showed  a  significant  correlation  between 
three  measures  of  mental  aptitude  and  the  outcome  variable  of 
salary  level  achieved. 

In  favor  of  situational  tests  over  mental  ability  tests, 
the  researchers  stated  that  when  mental  ability  as  measured 
by  a  paper-and-pencil  test  was  partialed  out  of  judged 
ability,  reliable  variance  still  remained.  They  found  that 
overall  and  combined  assessment  ratings  did  tend  to  be  higher 
than  the  correlations  for  any  individual  technique  (Bray  & 
Grant,   1966) . 

In  1967,  Bentz  (in  Campbell  et  al.,  1970)  summarized  the 
results  of  the  Sears,  Roebuck  Company  psychological  testing 
program.  In  relating  aptitude  and  intelligence  measures  to 
the  prediction  of  executive  effectiveness,  a  significant 
correlation  was  found.  Bentz  stated  that  the  successful 
administrators  were  superior  in  intellectual  endowment. 

Kraut  (1969) ,  while  working  for  IBM  Corporation,  sought 
to  determine  the  relationship  between  high  level  managers' 
promotional  success  and  measures  of  their  intellectual 
ability.  No  significant  relationships  were  found.  The 
researcher  cited  problems  with  the  predictor  tests  and  range 
restriction  of  the  sample  as  possible  reasons  for  the  results 
obtained. 

Wollowick  and  McNamara  (1969) ,  while  with  IBM  Corporation, 
sought  to  determine  the  validity  of  an  assessment  center 
approach  in  predicting  management  potential  and  to  determine 
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the  relative  value  of  the  components  of  the  program.  Within 
this  study,  they  found  that  two  measures  of  aptitude  and 
intelligence  were  not  significantly  related  to  the  outcome 
variable.  The  researchers  suggested  that  "it  may  be  possible 
to  consider  eliminating  the  paper-and-pencil  tests  not 
contributing  to  the  predictive  validity"   (p.   352) . 

Campbell  et  al.  (1970),  in  a  study  involving  the  Standard 
Oil  Company  of  New  Jersey,  attempted  to  discover  how  employees 
who  possess  the  potential  to  be  successful  in  management  could 
be  identified  early  in  their  careers.  Although  the 
intelligence  and  aptitude  measures  used  were  not  significantly 
related  to  the  outcome  criterion,  Laurent  (in  Campbell  et  al., 
1970)  concluded  that  successful  managers  "have  shown  a  total 
life  pattern  of  successful  endeavors"   (p.   169) . 

Moses  (1972)  studied  the  relationship  between  assessment 
and  subseguent  progress  in  management  for  personnel  in  the 
Bell  Telephone  System.  This  study  was  part  of  the  AT&T 
assessment  center  process,  but  not  a  part  of  the  original 
study  (Bray  &  Grant,  1966) .  A  significant  relationship  was 
found  between  aptitude  and  intelligence  measures.  The  study 
involved  a  large  sample  of  nonmanagement  personnel. 

Moses  and  Boehm  (197  5)  replicated  the  study  done  by  Moses 
in  1972,  but  the  sample  consisted  of  nonmanagement  females. 
A  significant  relationship  was  found  in  this  study.  The 
researchers  stated  that  "the  assessment  process  predicts  the 
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future  performance  of  women  as  accurately  as  it  does  that  of 
men"   (p.   529)  . 

Grimsley  and  Jarrett  (1975) ,  while  with  the  University  of 
Southern  California  and  California  State  University, 
respectively,  studied  the  relationship  between  past 
achievement  test  scores  and  criterion  measures  obtained  in 
administrative  positions.  The  significant  results  found  in 
this  concurrent  validity  study  led  the  researchers  to  conclude 
that  "the  differences  in  test  scores  of  more  or  less 
successful  managers  result  from  fundamental  differences  in 
mental  ability  and  personality  rather  than  the  influence  of 
on-the-job  experience"   (p.   226) . 

Huck  and  Bray  (1976)  studied  a  sample  of  non-management 
females  derived  from  the  AT&T  assessment  center  process. 
Their  purpose  was  to  test  the  validity  of  an  assessment  center 
process  on  a  population  different  from  the  population  of  the 
Management  Progress  Study  (Bray  &  Grant,  1966) .  Although  the 
relationship  between  aptitude  and  intelligence  measures  and 
the  criterion  variable  did  not  prove  to  be  significant,  the 
researchers  found  that  "judgments  made  by  assessment  center 
staffs  are  good  predictors  of  later  performance"   (p.   27) . 

Gantz,  Erickson,  and  Stephenson  (1977)  sought  to  determine 
why  some  persons  in  a  research  and  development  population 
achieve  promotion,  particularly  into  supervisory  or  managerial 
positions,  and  other  persons  do  not.  Measures  of  aptitude  and 
intelligence     were     not     significantly     related     to  either 
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promotion  rate  or  to  supervisor  ratings  in  this  concurrent 
study . 

Hinrichs  (1978)  evaluated  the  predictive  validity  of  the 
AT&T  assessment  center  process  and  compared  the  predictive 
accuracy  of  the  assessment  center  with  the  naturalistic 
management  evaluation.  No  significant  findings  were  obtained 
between  aptitude  and  intelligence  measures  and  administrative 
level  achieved.  The  researchers  found  that  "prediction  based 
upon  managerial  review  of  the  personnel  files  did  as  well  as 
the  assessment  center  after  8  years"   (p.   600) . 

Conclusions .  Restriction  of  range  was  acknowledged  as  the 
biggest  drawback  to  the  predictive  ability  of  aptitude  and 
intelligence  measures.  Preselection  of  the  samples  by 
management  tended  to  reduce  predictability  (Dicken  &  Black, 
1965) .  Intelligence,  as  measured  typically  by  verbal  ability 
tests,  is  a  fair  predictor  of  first-line  supervisory 
performance  but  not  of  higher-level  managerial  performance 
(Korman,  1968)  .  The  conclusions  drawn  by  most  researchers  in 
explaining  the  general  lack  of  a  significant  relationship 
between  aptitude  and  intelligence  measures  and  measures  of 
administrative  achievement  would  support  the  statement  that 
"measures  of  ability  and  interest  cannot  be  expected  to  make 
fine  discriminations"  (Dicken  &  Black,  1965,  p.  46). 
Personal  Interviews  as  a  Selection  Method 

"The  use  of  the  interview  as  a  device  for  appraising 
applicants  for  a  job  is  generally  regarded  with  a  good  deal 


of    suspicion    and    distrust    by     industrial  psychologists" 

(Ghiselli,    1966,    p.    389) .      A   factor  that  very   likely  has 

produced  disaffection  with  the  employment   interview  is  its 

nebulous  and  intangible  character.     The  interview  involves  a 

social  interaction  between  the  interviewer  and  the  applicant, 

thus   it  varies   substantially   in   form  and  content   from  one 

applicant  to  another  for  one  and  the  same  interviewer. 

In  attempting  to  improve  the  selection  interview, 
industrial  psychologists  have  sought  to  develop  sets 
of  rules  to  follow  in  order  to  improve  the  reliability 
and  validity  of  the  interview.  To  some  extent  these 
rules  are  based  upon  scattered  empirical  evidence,  but 
because  research  on  the  interview  is  so  very  limited 
they  are  more  commonly  based  upon  professional 
judgment  or  sheer  common  sense.  Mayfield's  (1964) 
attempt  to  integrate  the  pertinent  research  can  only 
be  described  as  heroic.  Yet  the  conclusions  and 
generalizations  he  arrives  at  are  but  weakly  founded. 
(Ghiselli,    1966,   p.  390) 

Early  studies  done  on  the  personal  interview  method  by 
Binet  and  Scott  (in  Arvey  &  Campion,  1982)  showed  low 
interrater  reliabilities.  Wagner  (1949) ,  in  a  summary  of 
interview  research,  noted  low  validities  in  most  cases.  In 
1964,  Mayfield  reviewed  all  interview  research  done  since 
Wagner  (1949)  and  found  a  median  validity  correlation 
coefficient  of  r  =  .27.  He  stated  that  this  value  was  not 
"particularly  high"   (Arvey  &  Campion,   1982,   p.  284). 

Research  used  in  this  study.  A  total  of  6  research 
studies  were  found  that  used  personal  interviews  as  a 
predictor  variable  and  met  the  other  delimitations  of  this 
dissertation.  These      studies      yielded      7  correlation 


coefficients  which  ranged  in  strength  from  r  =  .03  to  r  =  .35. 
Samples  ranged  in  size  from  122  to  8,885  persons. 

Ghiselli  (1966),  while  at  the  University  of  California, 
attempted  to  evaluate  the  validity  of  data  gathered  during 
personal  interviews  in  predicting  administrative  job 
proficiency.  The  independent  variable  was  the  personnel 
interview  used  in  a  corporation  over  a  17-year  period  in 
hiring  account  executives.  The  outcome  criterion  was  survival 
with  the  company  for  a  3 -year  period.  The  interview  was 
relatively  unstructured  and  included  no  guestions  that  were 
of  a  highly  personal  nature.  However,  some  guestions  that 
would  not  be  legal  today  in  a  public  employment  interview  were 
used.  A  significant  correlation  was  found  between  the  two 
variables  in  this  highly  range-restricted  sample.  Ghiselli 
thus  found  the  interview  to  have  "at  least  a  moderately 
substantial  validity"   (p.   394) . 

Grant  and  Bray  (1969)  studied  the  contributions  made  by 
the  interview  to  the  assessment  center  process,  and  also,  the 
relationships  between  interview  variables  and  the  progress 
criterion  of  salary  level  attained  after  9  years.  These 
relatively  unstructured  interviews  covered  job-related  and 
personal     topics.  The     interviewers     were  professional 

psychologists.  For  the  purpose  of  the  study,  the  researchers 
divided  the  interview  into  18  different  personality  variables 
and  related  each  variable  to  the  outcome  criterion.  The 
significant  relationship  found  showed  that  "the  interview  did 


produce  reliable  ratings  of  managerial  gualities  which  related 
significantly  with  ratings  made  on  the  basis  of  several  other 
technigues  and  with  advancement"   (p.   34) . 

Carleton  (1970) ,  while  with  the  Standard  Oil  Company,  Ohio 
(SOHIO)  assessment  program,  studied  the  relationships  of 
several  personnel  selection  methods  to  the  outcome  criterion 
of  supervisor  rating.  A  period  of  2.5  to  5  years  elapsed 
between  measurement  of  variables.  The  researcher  concluded 
that  "results  of  the  interview  report  appeared  particularly 
impressive  in  light  of  the  usual  criticism  of  the  interview 
as  an  assessment  technigue"   (p.   566) . 

Campbell  et  al.  (1970),  while  at  the  University  of 
Minnesota  and  Yale  University  (Lawler) ,  in  the  Standard  Oil 
Company  of  New  Jersey  assessment  center  study,  reviewed  the 
relationship  between  the  personal  interview  and  later  measures 
of  administrative  achievement.  The  outcome  criterion 
consisted  of  a  combination  of  performance  ratings  to  include 
salary  level  attained,  supervisor  ratings,  and  administrative 
level  achieved. 

Moses  (1972)  conducted  a  study  of  8,885  nonmanagement 
males.  These  men,  while  employed  by  the  Bell  System,  were 
interviewed  as  part  of  the  assessment  center  process.  A 
significant  correlation  was  found  with  the  criterion  of 
administrative  level  achieved. 

Turnage  and  Muchinsky  (1984)  conducted  a  study  based  upon 
data  from  a  one-day  supervisory  selection  program  (assessment 


center)  developed  by  a  large  manufacturing  firm.  The  subjects 
were  799  nonmanagement  employees  who  were  subsequently 
promoted  to  supervisory  positions.  The  individual  interview 
was  based  largely  on  the  candidate's  Questionnaire  responses, 
in  which  the  candidate  was  reguired  to  play  the  role  of  a 
supervisor  faced  with  discharging  an  employee.  In  this 
respect,  this  interview  method  overlaps  the  method  of  job- 
related  skills  indicators.  The  predictor  variable,  interview, 
was  correlated  separately  with  three  criteria:  salary  level 
attained,  supervisor  ratings,  and  promotion  rate.  Only  one 
resulting  coefficient  was  significant.  Based  on  these,  and 
other  findings  in  the  study,  the  researchers  concluded  that 
there  appeared  to  be  no  appreciable  relationship  between  how 
one  is  evaluated  in  an  assessment  center  and  how  one  performs 
on  the  job. 

A  number  of  reasons  were  surmised  by  the  researchers  for 
the  lack  of  correlation.  Methodological  factors  such  as  low 
criterion  reliability,  low  predictor  reliability,  severe 
restriction  of  range,  marked  skew  in  the  data,  procedural 
inconsistencies,  lack  of  comparability  across  assessment 
groups,  and  errors  in  data  collection,  were  cited  as  possible 
problems  within  this  concurrent  study. 

Conclusions .  The  structured  personal  interview,  in  which 
the  interviewer,  or  interviewers,  follow  a  set  schedule  of 
guestions,  has  consistently  shown  higher  reliabilities  than 
the    unstructured    interview    (Wright,     1969) .       In  addition, 
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interviews  conducted  by  a  board  or  panel  appear  to  be 
promising  as  a  means  of  enhancing  reliability  and  validity 
(Arvey  &  Campion,  1982).  Greater  validities  may  be  found  if 
researchers  first  decide  the  purpose  the  interview  is  intended 
to  serve.  It  has  been  suggested  that  the  personal  interview 
is  useful  in  determining  interpersonal  skills  and  motivation 
(Schmitt,  1976) ,  and  in  imparting  job  information  from  the 
interviewer  to  the  applicant   (Arvey  &  Campion,   1982) . 

The  employment  interview  continues  to  be  used  although 
organizational  psychologists  are  aware  of  the  findings 
concerning  the  method's  limited  reliability  and  validity. 
Arvey  and  Campion  stated  that  "interviewers  ignore  base  rate 
information,  do  not  pay  attention  to  disconf irming  evidence, 
and  over-depend  on  case  specific  information  in  making  their 
judgments"  (Arvey  &  Campion,  1982,  p.  316). 
Job-related  Skills  Indicators  as  a  Selection  Method 

Situational  methods  offer  the  potential  of  adding  to  the 
scope  of  human  characteristics  that  can  be  evaluated.  The 
rationale  behind  using  situational  exercises  is  that  they 
simulate  the  type  of  work  to  which  the  candidate  will  be 
exposed  and  allow  his  or  her  performance  to  be  observed  under 
somewhat  realistic  conditions.  Situational  tests  measure  more 
complex  or  dynamic  behavior  rather  than  aptitudes  or  traits 
isolated  by  more  traditional  psychometric  tests.  The  whole 
personality  is  observed  in  interaction  with  simulations  of  the 
future  job  environment   (Howard,   1974) . 
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The  most  commonly  used  situational  tests  of  job-related 
skills  for  administrative  positions  are  examined  here.  The 
in-basket  exercise  is  one  of  those  most  frequently  used  in 
assessment  centers.  The  candidate  is  faced  with  an 
accumulation  of  memos,  reports,  notes  of  incoming  telephone 
calls,  letters,  and  other  materials  supposedly  collected  in 
the  in-basket  of  the  job  he  or  she  is  to  take  over.  The 
candidate  is  asked  to  dispose  of  these  materials  in  the  most 
appropriate  manner  by  writing  letters,  notes,  self-reminders, 
agenda  for  meetings,  etc.  Ratings  of  performance  range  from 
subjective  evaluations  to  highly  standardized  checklists. 

In  the  leaderless  group  discussion,  the  participants  are 
given  a  discussion  question  and  are  instructed  to  arrive  at 
a  group  decision.  Topics  may  include  such  things  as  promotion 
decisions,  disciplinary  actions,  or  business  expansion 
problems.  Sometimes  participants  are  given  a  particular  point 
of  view  to  defend.  Personality  dimensions  such  as 
interpersonal  skills,  acceptance  by  the  group,  individual 
influence,  and  leadership  abilities  can  be  evaluated  in  this 
way. 

Management  games,  such  as  stock  market  tasks, 
manufacturing  exercises,  and  merger  negotiations  are  common 
job-related  selection  exercises.  Participants  are  asked  to 
solve  problems,  either  cooperatively  or  competitively.  These 
games  often  bring  out  leadership,  organizational  abilities, 
and  interpersonal  skills.    Some  games  also  permit  observations 
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under  stress,  especially  when  conditions  suddenly  change  or 
when  competition  stiffens. 

Research  used  in  this  study.  A  total  of  10  research 
studies  were  found  that  used  job-related  skills  indicators  as 
a  predictor  variable  and  met  the  other  delimitations  of  this 
dissertation.  These  10  studies  yielded  12  correlation 
coefficients  which  ranged  in  strength  from  r  =  .  02  to  r  =  .  49 . 
Samples  ranged  in  size  from  31  to  8,885  persons. 

In  1965,  Tenopyr  and  Ruch  (in  Campbell  et  al .  ,  1970) 
studied  the  relationship  between  job-related  skills  and  salary 
level  attained  in  a  concurrent  validity  study.  Subjects  were 
production  managers  at  North  American  Aviation.  They  were 
given  a  test  designed  to  assess  a  supervisor's  ability  to 
handle  human  relations  problems.  Although  a  significant 
finding  resulted,  the  researchers  concluded  that  this 
correlation  was  "useful"  but  not  "overwhelming"  (Campbell  et 
al. ,    1970,   p.    193) . 

Dicken  and  Black  (1965)  studied  the  validity  of  clinical 
interpretations  of  an  objective  test  battery  in  an  industrial 
setting.  A     sample     of     first-line     supervisors      in  a 

manufacturing  company  was  given  tests  of  job-related  skills. 
These  tests  were  administered  to  the  participants  individually 
and  in  groups.  A  significant  correlation  was  found  between 
job-related  skills  indicators  and  salary  level  attained  after 
3.5  years. 
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Bray  and  Grant  (1966)  found  that  job-related,  or 
situational,  selection  methods  had  considerable  influence  on 
the  judgments  of  the  assessor.  Job-related  skills  indicators 
of  a  sample  of  managers  were  related  to  salary  level  attained 
8  years  later.  A  significant  correlation  was  found.  The 
situational  exercises  used  consisted  of  an  in-basket  exercise, 
a  simulated  manufacturing  problem,  and  a  leaderless  group 
discussion.  The  researchers  concluded  that  the  job-related 
situational  exercises  contributed  enough  to  predictivity  to 
warrant  their  relatively  high  costs  in  time  and  money. 

Wollowick  and  McNamara  (1969)  studied  the  validity  of  an 
assessment  center  approach  in  predicting  management  potential 
and  to  determine  the  relative  values  of  the  components  of  the 
program.  One  component  of  the  assessment  center  consisted  of 
both  written  and  performance  tests  of  job-related  skills. 
These  tests  were  administered,  individually  and  in  groups,  to 
a  sample  of  low-  and  mid-management  men.  The  results  of  these 
tests  were  significantly  related,  approximately  three  years 
later,   to  administrative  level  achieved. 

Meyer  (1970),  while  at  General  Electric  Company,  studied 
the  relationship  between  a  job-related  skills  indicator,  the 
in-basket  exercise,  and  an  outcome  criterion  of  supervisor 
ratings.  A  demonstrated  inability  of  a  general  aptitude  test 
to  determine  which  employees  would  become  successful  unit 
managers  led  to  the  use  of  this  method.  This  exercise  was 
developed  by  John  Hemphill,   then  of  the  Educational  Testing 
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Service.  A  comparison  of  overall  score  on  the  skills 
indicator  test  and  the  outcome  variable  yielded  a  significant 
correlation  in  this  concurrent  study.  Meyer  concluded  that 
the  "performance  style  one  exhibits  in  handling  carefully 
selected,  but  true-to-life  *in-basket'  items  does  correlate 
with  demonstrated  on-the-job  performance  of  a  managerial 
position,  especially  the  ability  to  handle  the  planning  and 
administrative  aspects  of  the  job"   (p.   307) . 

Campbell  et  al .  (1970)  studied  the  relationship  between 
job-related  skills  indicators  and  a  criterion  measure  of 
administrative  achievement.  The  outcome  criterion  was  a 
combination  of  performance  ratings,  to  include  salary  level 
attained,  supervisor  ratings,  and  administrative  level 
achieved.  This  concurrent  validity  study  was  a  component  of 
the  Early  Identification  of  Management  Potential  (EIMP) 
assessment  program  of  the  Standard  Oil  Company  of  New  Jersey. 
A  subject  group  of  mid-  to  high-level  managers  showed  a 
significant  correlation  between  written  measures  of  job- 
related  skills  indicators  and  the  outcome  criterion. 

The  researchers  found  that  job-related  skills  indicators, 
in  the  form  of  a  management  judgment  test,  were  among  the  four 
best  measures  of  achievement.  However,  they  surmised  that  the 
high  correlation  was,  in  part,  attributable  to  possible 
criterion  contamination  due  to  the  managers'  perceptions  of 
their  own  career  success. 
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Moses  (1972)  studied  the  relationship  between  job-related 
skills  indicators  and  administrative  level  achieved  seven 
years  later.  The  job-related  skills  indicators  were 
performance-type  tests  by  which  assessors  determined  a 
subject's  organizing,  planning,  decision-making,  and 
leadership  skills.  A  large  sample  of  nonmanagement  men  showed 
a  significant  correlation  coefficient  between  the  two 
variables . 

Moses  and  Boehm  (1975)  studied  the  relationship  between 
several  aspects  of  performance  at  an  assessment  center  and 
subseguent  administrative  level  achieved.  A  large  sample  of 
nonmanagement  women  was  assessed  between  1963  and  1971.  A 
significant  correlation  was  found  in  this  predictive  study. 
This  study  was  not  part  of  the  landmark  AT&T  study  by  Bray  and 
Grant  (1966)  and  some  criterion  contamination  may  have 
occurred.  The  researchers  concluded  that  the  job-related 
skills  indicators  of  leadership,  decisionmaking,  organizing, 
and  planning  related  highly  with  the  outcome  criterion. 

Huck  and  Bray  (1976) ,  while  with  the  Wickes  Corporation 
and  AT&T,  respectively,  studied  the  power  of  the  AT&T 
assessment  center  to  predict  future  job  performance  of  female 
employees.  Subjects  were  non-management  personnel  of  Michigan 
Bell  Telephone.  The  independent  variable  of  job-related 
skills  indicators  consisted  of  performance  tests  of 
leadership,  decision-making,  planning,  and  organizing.  Some 
criterion  contamination  was   acknowledged.      The  researchers 
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concluded  that  the  assessment  center  process  was  as  valid  a 
selection  method  for  women  as  for  men. 

Turnage  and  Muchinsky  (1984),  while  at  Iowa  State 
University,  conducted  a  predictive  study  based  upon  data  from 
a  one-day  supervisory  selection  program  (assessment  center) 
developed  by  a  large  manufacturing  firm.  The  subjects  were 
employees  who  were  subseguently  promoted  to  supervisory 
positions.  An  overall  significant  correlation  was  found 
between  job-related  skills  indicators  and  these  methods  of 
measuring  subseguent  job  performance  combined.  This  was  one 
of  the  few  significant  results  of  this  study. 

Conclusions.  Though  much  more  expensive  and  time 
consuming  to  administer  than  paper-and-pencil  tests  and 
guestionnaires,  the  need  to  find  ways  of  evaluating 
characteristics  not  covered  by  the  latter  is  sufficient  to 
warrant  extensive  experimentation  with  relatively  elaborate 
technigues  (Bray  &  Grant,  1966) .  There  has  been  relatively 
little  research  on  the  relationship  between  how  a  manager 
behaves  in  a  game  and  his  behavior  in  an  actual  decision 
situation.  The  behavior  of  a  manager  in  a  business  game  may 
be  guite  different  from  his  behavior  on  the  job  where  the 
rewards  and  punishments  are  much  larger.  Further,  business 
games  tend  to  de-emphasize  the  interpersonal  dimension  in 
managerial  performance,  whereas  many  management  jobs  appear 
to  emphasize  it  heavily   (Lawler,   1967,   p.  370). 
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Psychological  Attribute  Indicators  as  a  Selection  Method 

McGregor  (1960)   reported  that  the  greatest  single  factor 

that  apparently  influences  superior  and  inferior  performance 

by    supervisory    people    was    to    be    found    in    the    area  of 

personality  variables.     According  to  Nash   (1966) , 

it  is  reasonable  to  expect  that  the  vocational 
interests  of  a  manager  might  be  related  to  the 
effectiveness  of  his  job  performance.  His  enthusiasm, 
effort,  and  level  of  job  satisfaction  may  be  largely 
determined  by  how  interested  he  is  in  his  work  and 
associates.      (p.  250) 

According  to  Edel  (1968),  for  executive  and  managerial 
positions,  personality  characteristics  may  well  be  more 
important  to  success  than  skill  or  technical  know-how  (p. 
231).  To  point  up  this  importance,  Micherson  (in  Edel,  1968) 
surveyed  79  large  and  small  business  corporations  and  reported 
that,  of  those  executives  who  failed,  over  70%  did  so  because 
of  some  flaw  in  their  personality  rather  than  from  a  lack  of 
ability. 

A  wide  variety  of  research  on  personality  tests  of  various 
types  has  been  conducted.  This  work  has  been  successful  in 
defining  problems  and  has  contributed  to  the  overall 
understanding  of  personality  adjustment.  Three  problems  in 
the  use  of  personality  tests  in  occupational  prediction 
deserve  special  mention:  (a)  the  ease  with  which  test  scores 
can  be  distorted  by  a  test-wise  applicant  to  portray  the  type 
of  personality  desired,  (b)  the  lack  of  reliability  displayed 
by  many  personality  measures,   and   (c)   the  failure  to  design 


54 

the  tests  specifically  for  purposes  of  occupational  prediction 
(Edel,  1968). 

Early  studies  gave  mixed  validity  results.  Knauft  (1949) 
studied  a  sample  of  3  3  bakery  shop  managers  and  a  correlation 
of  r  =  .39  was  found,  but  the  personality  test  used  was  later 
withdrawn  from  the  market  (Korman,  1968) .  Comrey  and  High 
(1955)  tested  the  validity  of  some  ability  and  interest 
scores.  Using  a  sample  of  over  200  production  supervisors, 
scores  on  several  preference  records  and  vocational  scales 
were  compared  to  objective  performance  data.  All  correlations 
except  one  were  insignificant.  La  Gaipa  (1960)  studied  the 
validity  of  certain  personality  traits  of  over  400  naval 
officer  candidates  compared  to  later  supervisor  ratings.  Only 
one  of  several  correlations  was  significant.  MacKinney  and 
Wolins  (1960)  studied  the  relationship  between  two  interest 
measures  and  several  later  measures  of  performance  using  three 
overlapping  samples  of  supervisors.  Results  were  inconsistent 
random  patterns  of  significant  and  insignificant  correlations. 
Mixed  results  were  also  found  by  Robbins  and  King  (1961)  in 
four  samples  of  sales  managers. 

Research  used  in  this  study.  A  total  of  22  research 
studies  were  found  which  used  psychological  attribute 
indicators  as  a  predictor  variable  and  met  the  other 
delimitations  of  this  dissertation.  These  studies  yielded  37 
correlation  coefficients  which  ranged  in  strength  from  r  =  .01 
to  r  =  .47.     Samples  ranged  in  size  from  20  to  1,375  persons. 
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Fleishman  and  Peters  (1962) ,  while  at  Yale  University, 
studied  the  relationships  between  psychological  attribute 
indicators  and  managerial  achievement.  The  resulting 
correlation  was  not  significant.  A  small  sample  was  used  in 
this  concurrent  study.  The  researchers  found  that  (a) 
managers  who  rated  highly  on  "conformity"  were  less  valued  by 
senior  raters,  (b)  the  leadership  attitudes  of  consideration 
and  structure  were  not  mutually  exclusive,  and  (c)  top 
management  tended  to  identify  the  effectiveness  of  subordinate 
managers  with  the  effectiveness  of  their  superiors. 

Hicks  and  Stone  (1962)  explored  the  relationship  between 
psychological  attribute  indicators  and  supervisors'  ratings 
in  this  concurrent  study.  A  nonsignificant  correlation  was 
found  for  the  sample  of  shop  foremen  and  engineering 
supervisors.  The  researchers  concluded  that  there  may  be 
certain  basic  characteristics  which  the  successful  managers 
possess  regardless  of  their  areas  of  specialization.  They 
described  the  personality  of  the  successful  manager  as  one  of 
emotional  strength  in  a  person  who  views  things  from  a  broad, 
theoretical  point  of  view,  avoiding  over  involvement  in 
detail . 

Goodstein  and  Schrader  (1963),  while  with  the  University 
of  Iowa  and  the  Civilian  Personnel  Field  Agency,  Ordnance 
Field  Activity,  United  States  Army,  respectively,  studied  the 
validity  of  a  personality  inventory  in  identifying  those 
personality   characteristics   associated  with  managerial  and 


supervisory  success  in  a  large  military  industrial 
organization.  In  their  concurrent  study  of  male  civilian 
managers  working  for  the  Army,  the  variable  of  psychological 
attribute  indicators  was  compared  to  the  criterion  of 
administrative  level  achieved.  A  significant  relationship  was 
found. 

The  researchers  stated  that  the  results  suggested  there 
may  be  significant  differences  in  personality  characteristics 
not  only  between  managers  and  nonmanagers  but  also  among 
managers  at  different  levels  of  responsibility.  They 
purported  that  success  in  first-line  supervision  may  be 
determined  mainly  by  technical  skill  and  knowledge  which  is 
relatively  independent  of  personality  factors  whereas,  at  the 
upper  levels  of  management,  such  personality-related  variables 
as  organizing,  directing,  planning,  and  decision  making  become 
important. 

Williams  &  Harrell  (1964)  attempted  to  discover  which 
personality  factors  were  related  to  business  success.  In 
their  predictive  study  they  used  the  salary  level  attained 
and  administrative  level  achieved  of  Stanford  MBA  graduates 
15  to  31  years  after  graduation.  The  overall  comparison  of 
psychological  attributes  to  later  achievement  was 
insignificant.  However,  there  was  a  significant  positive 
relationship  between  "success  and  the  score  on  the 
Masculinity-Femininity  scale  {of  the  Strong  Vocational 
Interest  Blank},    indicating  that  those  individuals  with  the 
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stronger  masculine  interests  have  a  somewhat  better  chance  of 
success"   (Williams  &  Harrell,   1964,  p.  167). 

Dicken  and  Black  (1965)  studied  the  predictive  validity 
of  psychometric  evaluation  for  the  selection  of  supervisors. 
Predictor  variables  were  two  well-known  personality 
inventories.  Two  criteria,  salary  level  attained  and 
promotion  rate,  both  measured  3.5  to  7  years  after  testing, 
were  used.  Insignificant  correlation  coefficients  were  found 
for  each  of  the  two  samples.  The  researchers  cited 
restriction  of  range  as  a  problem  in  studies  of  this  type  by 
preselection  of  the  samples  by  management.  They  stated  that 
"measures  of  ability  and  interest  cannot  be  expected  to  make 
fine  discriminations"   (p.   46) . 

Bray  and  Grant  (1966)  explored  the  relationship  between 
psychological  attribute  indicators  and  salary  level  attained 
8  years  later.  Subjects  were  low-level  managers.  Although 
an  overall  nonsignificant  correlation  was  found  between  the 
variables,  several  personality  traits  such  as  lack  of 
passivity  and  control  of  feelings  were  positively  correlated 
with  success  in  management. 

Nash  (1966) ,  while  at  the  University  of  Maryland,  sought 
to  discover  the  relationships  between  a  manager's  vocational 
interests  and  the  effectiveness  of  his  job  performance.  This 
concurrent  validity  study  used  corporation  managers  as 
subjects.  The  correlation  coefficient  between  psychological 
attribute  indicators  and  supervisor  ratings  was  found  to  be 


58 

significant.  Nash     concluded     that     "effectiveness  is 

significantly  related  to  the  vocational  interest  patterns  of 
managers"   (p.  254)  . 

In  1967,  Bentz  (in  Campbell  et  al.,  1970)  summarized  a 
series  of  studies  which  he  had  conducted  within  the 
Psychological  Research  and  Services  Section  of  Sears,  Roebuck 
Company.  In  this  study  a  significant  relationship  was  found 
between  psychological  attribute  indicators  and  promotion  rate 
within  Sears.  Bentz *s  conclusion  was  that  the  Sears  managers 
accomplish  their  goals  because  they  are  superior  "in 
intellectual  endowment,  social  competence,  and  emotional 
stamina"  (Campbell,  et  al.  1970,  p.  187).  Campbell  et  al. 
(1970)  stated  that  there  was  truth  to  Bentz ' s  statement,  but 
that  it  was  not  strongly  supported  by  the  findings  of  his 
concurrent  validity  study. 

Cummin  (1967) ,  while  at  Harvard,  sought  to  discover  if  the 
"more  successful"  executives  would  display  motivation  toward 
achievement,  power,  and  autonomy,  and  the  "less  successful" 
executives  would  show  motivation  toward  affiliation, 
aggression,  and  deference  in  a  personality  inventory.  His 
concurrent  validity  study  of  business  persons  compared 
psychological  attribute  indicators  to  current  salary  level 
attained.  He  found  a  significant  correlation.  Cummin 
concluded  that  "the  successful  executive  is  one  who  is 
determined  to  maintain  a  high  standard  of  excellence  in  his 
work,  and  to  assume  greater  responsibilities  and  more  control 
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over  his  environment  as  he  advances  within  the  organization" 
(P-   81)  . 

Grant,  Katkovsky,  and  Bray  (1967),  while  at  AT&T  and 
Fordham  University  (Katkovsky) ,  studied  the  contributions  of 
projective  techniques  to  the  assessment  of  management 
potential.  The  study  used  data  collected  in  the  Bell  System 
Management  Progress  Study  (Bray,  19  64) .  A  comparison  of 
psychological  attribute  indicators  to  salary  level  attained 
eight  years  later  showed  a  nonsignificant  correlation.  The 
researchers  found  that  within  this  method  several  of  the 
projective  variables  were  reliably  related  to  salary  progress. 
Those  variables  were  dependence  and  subordinate  roles  (with 
negative  relationships) ,  achievement  motivation,  and 
leadership  role. 

In  a  concurrent  validity  study,  Ghiselli  (1968)  evaluated 
motivational  factors  in  the  success  of  managers.  He 
administered  a  personality  inventory  to  several  samples  of 
corporation  middle  management  personnel.  Psychological 
attribute  indicators  related  significantly  with  supervisor 
ratings . 

As  compared  with  the  employed  population  as  a  whole, 
Ghiselli  found  that  persons  in  middle  management  positions 
appeared  to  have  a  substantially  lower  desire  for  security  and 
for  financial  reward  and  a  higher  desire  for  self- 
actualization.  They  did  not  differ  from  the  employed 
population  in  the  desire  for  power  over  others. 
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Edel  (1968) ,  while  working  for  the  Department  of  Defense, 
studied  the  relationship  between  a  psychological  "need  for 
success"  and  managerial  performance.  He  found  a  significant 
correlation  in  this  concurrent  study.  Edel's  study  used  one 
test  to  measure  need  for  success.  Reliability  data  had  not 
yet  been  provided  for  this  test. 

In  1968,  Miner  (in  Miner,  1977),  while  at  Georgia  State 
University,  studied  the  relationship  between  measures  of 
psychological  attributes  of  school  administrators  and 
separate,  concurrent,  outcome  criteria  of  supervisor  ratings 
and  salary  level  attained.  He  found  no  significant 
correlations.  He  concluded  that  managerial  motivation  was  not 
rewarded  in  that  school  system. 

Campbell  et  al.,  (1970)  conducted  the  Early  Identification 
of  Management  Potential  study  for  Standard  Oil  Company  of  New 
Jersey.  Within  this  study  they  compared  measures  of 
psychological  attributes  to  a  combination  of  concurrent 
outcome  criteria  to  include  salary  level  attained,  supervisor 
ratings,  and  administrative  level  achieved.  Subjects  were 
mid-  to  high-level  corporation  managers.  An  insignificant 
correlation  was  found.  The  researchers  concluded  that  these 
indicators  of  temperament  showed  no  useful  relationship  with 
any  of  the  effectiveness  measures. 

Harrell  and  Harrell  (1974)  conducted  a  predictive  study 
in  order  to  determine  predictors  of  administrative 
achievement.    While  working  for  the  Office  of  Naval  Research, 
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they  found  a  significant  correlation  between  psychological 
attribute  indicators  and  salary  level  attained  10  years  later. 
Subjects  were  former  MBA  graduates  who  had  achieved  management 
positions  in  corporations. 

Grimsley  and  Jarrett  (1975)  studied  the  relationship 
between  managerial  achievement  and  test  measures  obtained  in 
the  employment  situation.  This  concurrent  study  used  two 
samples  of  mid-  and  top-level  corporation  managers.  The 
results  of  several  measures  of  psychological  attributes  were 
compared  to  the  outcome  criteria  of  administrative  level 
achieved.  Insignificant  correlation  coefficients  were  found 
with  each  of  the  two  samples.  The  researchers  concluded  that 
"the  differences  in  test  scores  of  more  or  less  successful 
managers  result  from  fundamental  differences  in  mental  ability 
and  personality  rather  than  the  influence  of  on-the-job 
experience"   (p.   226) . 

Gantz,  Erickson,  and  Stephenson  (in  Miner,  1977)  sought 
to  determine  if  certain  psychological  attributes  could  be 
causes  of  managerial  success.  Using  a  sample  of  research 
scientists  and  engineers  in  a  federally  funded  laboratory, 
several  measures  of  psychological  attributes  were 
administered.  These  proved  to  be  insignificantly  related  to 
both  supervisors'  ratings  and  promotion  rate.  The  researchers 
concluded  that  perceived  creativity  was  rewarded  by  promotion 
into  leadership  roles,  including  formal  managerial  roles  where 


62 

managerial  skill  was  needed,  and  actual  or  potential 
leadership  ability  was  not  rewarded. 

Two  consequent  studies  by  Gantz,  Erickson,  and  Stephenson, 
were  also  reported  in  Miner  (1977).  In  the  first,  a  sample 
of  former  research  scientists  was  followed  up  5.3  years  after 
testing  and  showed  a  significant  correlation  between  measures 
of  psychological  attributes  related  to  motivation  to  manage 
and  promotion  rate.  As  some  of  the  subjects  may  have 
experienced  grade  level  changes  based  in  part  on  research 
competence,  the  success  index,  promotion  rate,  may  not  have 
been  based  entirely  on  managerial  competence.  The  researchers 
concluded  that  the  borderline  level  of  the  correlation 
indicated  that  further  analyses  dealing  with  the  predictive 
power  of  the  role-motivation  theory  ought  to  be  conducted. 

Using  a  second  sample  consisting  of  top  sales  persons  and 
marketing  managers,  the  researchers  conducted  a  similar  study 
with  four  years  intervening  between  variables.  The 
correlation  was  significant.  The  researchers  stated  that, 
within  this  department  of  the  same  company,  "promotion  into 
the  higher  grade  levels  was  based  entirely  on  managerial 
competence"  (Miner,  1977,  p.  31).  The  conclusion  drawn  was 
that  the  motives  measured  by  the  particular  psychological 
attributes  indicator  used  did  serve  as  a  cause  of  subsequent 
managerial  accomplishment  (Miner,   1977) . 

Miner   (1977),    in  a  concurrent  validity  study,    sought  to 
discover    the    relationships    between    different    measures  of 
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psychological  attributes  and  two  measures  of  administrative 
achievement.  Using  a  sample  of  business  personnel  managers, 
he  found  insignificant  relationships  with  both  administrative 
level  achieved  and  with  salary  level  attained.  Although 
neither  correlation  was  significant,  Miner  stated  that  "the 
overall  measures  of  motivation  to  manage  are  quite 
consistently  related  to  the  occupational  success  indexes"  (p. 
74)  . 

In  a  separate  study  by  Miner  (1977),  faculty  members  and 
administrators  in  three  different  business  schools  were  given 
an  inventory  of  psychological  attributes.  Concurrently,  each 
subject's  score  was  compared  with  his  or  her  administrative 
level  attained.  Miner  reported  no  significant  relationship, 
and  "if  anything,  the  administrators  have  less  motivation  to 
manage  than  the  regular  faculty,  although  the  difference  does 
not  approach  significance"   (p.   51)  . 

McClelland  and  Boyatzis  (1982),  while  at  Harvard 
University  and  McBer  &  Company,  respectively,  studied  the 
relationship  between  psychological  attribute  indicators  and 
long-term  achievement  in  management.  Subjects  were  entry- 
level  managers  who  were  part  of  the  AT&T  assessment  center 
study.  A  significant  correlation  was  found  between  the 
predictor  variable  and  administrative  level  achieved  after  16 
intervening  years.  The  researchers  concluded  that  a  high  need 
for  achievement  was  associated  with  managerial  success  at 
lower    levels    of    management,     while    at    higher    levels  of 
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management  persons  with  a  need  for  influencing  others 
dominated. 

Stahl  (1983) ,  while  at  Clemson  University,  used  a  measure 
of  psychological  attributes  to  test  the  hypothesis  that  high 
managerial  motivation  consisted  of  high  personal  needs  for 
achievement  and  power.  He  found  a  significant  correlation 
with  each  of  two  outcome  criterion  variables,  supervisor 
ratings  and  administrative  level  achieved.  The  researcher's 
conclusions  were  drawn  from  this  and  other  data  collected  from 
other  samples  using  the  same  methodology.  He  concluded  that 
"there  was  a  higher  proportion  of  subjects  with  high 
managerial  motivation  among  the  managers  than  among  the  non- 
managers,"  and  that  "there  was  a  higher  proportion  of  managers 
with  high  managerial  motivation  among  the  promoted  managers 
than  among  the  non-promoted  managers"  (p.  786) .  Stahl 
stressed  the  need  for  researchers  to  implement  longitudinal 
validation  studies  to  confirm  these  results. 

Turnage  and  Muchinsky  (1984)  compared  the  predictive 
validity  of  assessment  center  evaluations  versus  traditional 
measures  in  forecasting  supervisory  job  performance.  A  sample 
of  employees  who  were  subsequently  promoted  to  supervisory 
positions  within  a  large  manufacturing  firm  was  used. 
Correlation  coefficients  between  findings  of  psychological 
attribute  indicators  and  three  separate  outcome  variables 
yielded  no  significant  relationships.  The  researchers  cited 
many  possible  methodological  shortcomings  of  the  study  that 
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could  have  contributed  to  the  general  lack  of  significant 
findings . 

Conclusions .  Results  obtained  by  the  studies  reviewed 
herein  suggest  that  there  is  evidence  supporting  the 
proposition  that  effective  managers  have  identifiable 
interests  which  distinguish  them  from  less  effective  managers. 
However,  most  motivation  studies  have  used  concurrent  validity 
as  the  research  method.  Predictive  validity  studies  that 
assess  the  dynamic  nature  of  personality  may  be  needed. 
According  to  Miner  (1978) ,  "the  greatest  research  needs  seem 
to  be  for  additional  longitudinal  studies  of  the  relationships 
between  motivation  to  manage... and  success"   (p.  751). 

Another  limitation  of  interest  measures  is  that  they  have 
been  demonstrated  to  be  fakeable.  Nash  suggested  that  future 
research  should  focus  on  the  impact  of  fakability  on  the 
actual  use  of  measured  interests  in  selection  programs  (1965, 
p. 34)  . 

Campbell  et  al.  (1970)  addressed  achievement  and  power  for 
managers.  Concerning  job/task  analysis  of  what  effective 
managers  actually  do,  they  listed  the  freguency  of  behavior 
aimed  at  influencing  others  (power)  and  the  freguency  of 
behavior  concerned  with  setting  and  accomplishing  goals 
(achievement) .  They  remarked  that  "better  managers  tend  to 
show  a  lifetime  pattern  of  high  achievement,  power,  and 
economic  motivation"   (p.   361) . 
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One  of  the  most  recent  and  extensive  treatments  of  the 

relationships  of  achievement  to  power  is  presented  in  Veroff 's 

book  of  readings  in  honor  of  McClelland. 

Achievement  motivation  directs  people  to  meeting 
socialized  standards  of  excellent  performance  and  thus 
to  highly  efficient  task-centered  strivings,  whereas 
power  motivation  directs  people  to  doing  whatever 
draws  most  attention  to  their  own  effect  on  the  world. 
The  two  motives  seem  to  be  fused  in  instances  where 
the  standard  of  excellence  is  to  win  in  a  social 
competitive  activity  or  to  solve  a  problem  that  will 
be  given  a  great  deal  of  recognition.  (Veroff,  1982, 
p.  100) 

Value  Systems  Assessments  as  a  Selection  Method 

The    selection    method    of    values     systems  assessments 

measures    the    strength    of    a    person's    economic,  aesthetic, 

social,   political,   and  religious  concerns   (Hinrichs,    1978) . 

According  to  England  and  Lee  (1974) , 

a  personal  value  system  is  viewed  as  a  relatively 
permanent  perceptual  framework  which  shapes  and 
influences  the  general  nature  of  an  individual's 
behavior.  Values  are  similar  to  attitudes  but  are 
more  ingrained,  permanent,  and  stable  in  nature;  they 
are  also  more  general  and  less  tied  to  any  specific 
referent  than  is  the  case  with  many  attitudes, 
(p.  412) 

Most  personnel  selection  research  studies  which  dealt  with 
the  independent  variable  of  psychological  attribute  indicators 
verged  on  or  overlapped  the  selection  methods  using  values  as 
an  indicator.  The  studies  used  in  this  section  were  those 
that  this  researcher  determined  were  predominantly  concerned 
with  values  as  defined  above. 

Research  used  in  this  study.  A  total  of  4  research 
studies  were  found  which  used  value  systems  assessments  as  a 
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predictor  variable  and  met  the  other  delimitations  of  this 
dissertation.  These      studies      yielded      5  correlation 

coefficients  which  ranged  in  strength  from  r  =  .10  to  r  =  .32. 
Samples  ranged  in  size  from  30  to  1,375  persons. 

In  1944,  the  Psychological  Research  and  Services  Section 
of  Sears,  Roebuck  Company  established  a  psychological  testing 
program.  Over  a  number  of  years  researchers  within  this 
program  conducted  a  series  of  investigations  into  the 
prediction  of  executive  effectiveness  in  the  Sears 
organization.  A  predictive  study  was  accomplished  by  Bentz 
in  1967  (in  Campbell  et  al.,  1970).  A  span  of  11  to  17  years 
elapsed  between  the  measurement  of  values  and  collection  of 
the  criterion  variable,  promotion  rate.  A  significant 
correlation  was  found. 

England  and  Lee  (1974),  while  at  the  University  of 
Minnesota,  studied  the  relationship  between  managerial  values 
and  managerial  success  in  several  countries.  The  portion  of 
their  study  that  dealt  with  managers  in  the  United  States  was 
used  in  this  dissertation.  Managerial  success  was  defined  as 
a  measure  of  salary  level  attained  in  this  concurrent  study. 
The  correlation  of  this  measure  with  a  written  survey  of 
values  was  significant  for  a  sample  of  corporation  directors 
and  high  level  executives.  The  researchers  concluded  that  the 
value  patterns  were  predictive  of  managerial  success  and  could 
be  used  in  selection  and  placement  decisions.  The  general 
pattern    that    emerged    from    the    study    indicated    that  more 
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successful  managers  appeared  to  have  pragmatic,  dynamic, 
achievement  oriented  values,  whereas  less  successful  managers 
had  more  static  and  passive  values,  corresponding  to  a  desire 
for  organizational  stasis  rather  than  an  organization  in  flux 
(England  &  Lee,    1974) . 

Grimsley  and  Jarrett  (1975)  studied  the  "effectiveness  of 
a  particular  methodological  approach  which  can  be  used  in 
analyzing  data  gathered  in  the  process  of  assessing  managerial 
applicants  in  the  employment  situation"  (p.  215) .  Two  sample 
groups  of  mid-  and  top-level  corporate  managers  were  used  in 
this  concurrent  study.  These  managers  came  from  more  than  100 
different  companies  and  many  industries  in  various  parts  of 
the  United  States.  All  were  administered  a  "study  of  values" 
test  and  test  scores  were  compared  with  administrative  level 
achieved.  The  results  were  not  significant.  The  findings  of 
this  study  led  the  researchers  to  conclude  that  the 
"differences  in  test  scores  of  more  or  less  successful 
managers  result  from  fundamental  differences  in  mental  ability 
and  personality  rather  than  the  influence  of  on-the-job 
experience"   (p.   226) . 

Hinrichs  (1978),  while  with  Management  Decision  Systems, 
Inc.,  conducted  an  8-year  followup  study  of  the  IBM 
Corporation  assessment  center.  Within  this  study,  the  results 
of  a  study  of  values  were  correlated  to  the  outcome  variable 
of  administrative  level  achieved.  The  resulting  correlation 
coefficient  of  r  =   .26  was  not  significant.     High  scores  on 
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political  and  economic  subsets  of  the  study  of  values  were 
positively  related  to  progression  in  formal  business 
hierarchy,  while  scores  on  the  religious  and  social  scales 
related  negatively. 

Conclusions.  The  few  research  findings  available  on 
values  as  a  selection  method  consistently  show  that  certain 
economic  and  political  beliefs  correlate  positively,  and 
strong  social  and  religious  beliefs  correlate  negatively,  with 
high  administrative  achievement  (Grimsley  &  Jarrett,  1975) . 
England  and  Lee  (1974)  found  that  "more  successful"  managers 
favored  an  achievement  orientation  and  preferred  an  active 
role  in  interaction  with  other  individuals  instrumental  to 
achievement  of  the  manager's  organizational  goals,  while  "less 
successful"  managers  had  values  associated  with  a  static  and 
protected  environment  in  which  they  took  relatively  passive 
roles  (pp.  418-419) .  According  to  Hinrichs  (1978)  ,  those 
traits  or  characteristics  that  seem  to  reflect  a  degree  of 
social  awareness  tend  to  be  detrimental  to  success  in 
administration.  "The  flavor  is  one  of  the  'hard  charging, 1 
perhaps  somewhat  socially  insensitive  and  upward  mobile 
individual"   (p.   600) . 

Biographical  Information  as  a  Selection  Method 

Biographical  information,  personal  history  data  about  an 
employee,  is  most  often  furnished  an  employer  in  the  form  of 
an  application  blank  or  a  resume.  According  to  Levine  and 
Flory  (1975) ,  the  most  widely  used  selection  technique  is  the 


evaluation  of  the  job  application  blank  or  the  resume.  These 
are  used  to  determine  whether  an  individual  meets  minimum 
qualifications  for  a  position.  If  these  qualifications  are 
not  met,  the  applicant  is  no  longer  considered.  In  some  cases 
this  is  the  only  information  an  employer  receives  about  a 
prospective  employee,  but  in  most  cases  the  application  is 
followed  by  an  interview,  written  test,  or  some  additional 
means  of  assessment. 

To  provide  some  idea  of  the  frequency  of  use  of 
applications,  Carlson  (in  Levine  &  Flory,  1975)  posited  that 
if  an  arbitrary  assumption  were  made  that  50%  of  those  filing 
applications  or  resumes  are  ultimately  interviewed,  then  over 
1,000,000,000  applications  and  resumes  per  year  are  prepared 
and  screened  in  the  United  States.  Yet,  despite  its 
universality,  the  evaluation  of  the  validity  of  biographical 
information  for  personnel  selection  and  placement  has  not  been 
researched  in  a  systematic  fashion  to  any  great  extent. 

Research  has  been  done  on  the  empirically  weighted 
application,  usually  for  predicting  turnover.  More  recent 
research  on  application  evaluation  has  focused  on  the  factors 
affecting  the  rater's  quality  of  evaluation  of  the  information 
rather  than  the  quality  of  the  method  of  evaluation  per  se. 
According  to  Spencer  and  Worthington  (1952)  and  Peck  and 
Parsons  (1956) ,  the  few  studies  dealing  with  method  of 
evaluation  other  than  empirical   weighing  have  demonstrated 


that  projective  evaluation  of  application  blank  information 
could  validly  predict  performance  and  tenure. 

A  trend  in  selection  research  in  recent  years  has  been  the 
increasing  use  of  personal  background  variables  in  the 
prediction  of  occupational  success.  Such  variables  are 
considered  to  have  several  advantages  for  this  purpose.  The 
most  frequently  cited  advantages  in  utilizing  these  kinds  of 
variables  are  that  (a)  they  are  less  threatening  than  the 
items  on  typical  personality  inventories  and  thus  are  less 
subject  to  "facade"  effects,  and  (b)  the  behaviors  described 
by  these  items  are  often  reflections  of  attitudinal  and 
personality  variables. 

Results  of  past  studies  of  the  relationship  between 
biographical  information  and  administrative  achievement  have 
been  mixed.  Vernon  (1950) ,  using  civil  service  managers,  and 
La  Gaipa  (1960) ,  using  Naval  officer  candidates,  found  no 
significant  relationships.  The  findings  of  Riccuiti  (1955), 
with  U.S.  Naval  officers,  Meyer  (1956),  with  firstline 
supervisors,  and  MacKinney  and  Wolins  (19  60)  with  supervisors, 
found  no  consistent  relationships.  Significant  relationships 
were  found  by  Haggerty  (1953),  at  the  U.S.  Military  Academy, 
Soar  (1956) ,  using  service  station  managers,  and  Scollay 
(1957) ,  with  promotion  managers. 

Childs  and  Klimoski  (1986),  in  a  study  which  involved 
employees  in  both  management  and  nonmanagement  positions 
investigated  the  validity  of  a  biographical  inventory  in  the 
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prediction  of  occupational  success.  They  found  that  positive 
social  orientation,  interpersonal  confidence,  and  educational 
achievement  were  positively  related  to  their  outcome  measures 
of  "job  and  career  success"   (p.  7) . 

Research  used  in  this  study.  A  total  of  8  research 
studies  were  found  which  used  biographical  information  as  a 
predictor  variable  and  met  the  other  delimitations  of  this 
dissertation.  These     studies     yielded     11  correlation 

coefficients  which  ranged  in  strength  from  r  =  .05  to  r  =  .57. 
Samples  ranged  in  size  from  30  to  799  persons. 

In  1963,  Haggerty  (in  Mayfield,  1970)  studied  the 
relationships  between  certain  predictor  variables  and 
achievement  of  U.S.  Army  officers.  In  this  predictive  study 
using  a  sample  of  officer  cadets,  the  researchers  found  a 
nonsignificant  correlation  between  biographical  information 
and  a  combination  of  achievement  measures. 

Williams  and  Harrell  (1964)  sought  to  determine  which,  if 
any,  of  a  number  of  factors  were  related  to  business  success. 
The  independent  variable  in  this  predictive  study  was  that  of 
biographical  information  available  to  an  employer  at  the  time 
of  a  student's  graduation  from  business  school.  Salary  level 
attained,  and  administrative  level  achieved,  measured  many 
years  after  graduation,  were  the  criteria  of  success. 
Insignificant  correlations  were  found  for  this  sample  of 
Stanford  MBA  graduates.  A  subset  of  this  information  that  was 
significantly  and  positively  related  to  later  achievement  was 


a  person's  previous  participation  as  a  leader  in  organizations 
on  campus. 

Kotula  and  Haggerty  in  1966  (in  Korman,  1968)  studied  the 
relationship  between  biographical  information  given  on  written 
personal  history  blanks  and  supervisor  ratings.  Two  groups 
of  Army  officers  were  studied.  Only  one  coefficient  was 
reported,  r  =  .17,  which  was  significant  at  the  .05  level. 

Campbell  et  al.  (1970)  attempted  to  discover  how  to 
identify,  early  in  their  careers,  those  employees  who  possess 
the  potential  to  be  successful  in  management.  The  assessment 
procedure  in  this  concurrent  study  contained  a  background 
biographical  survey  the  scores  from  which  were  correlated  to 
an  "Overall  Success  Index",  a  combination  of  dependent 
variables.  A  sample  of  mid-  to  high-level  corporation 
managers  was  used.  A  significant  correlation  was  found 
between  biographical  information  and  the  outcome  criterion  of 
achievement.  Laurent,  one  of  the  researchers  involved  with 
this  study,  concluded  that  "successful  executives  in  the 
Standard  Oil  of  New  Jersey  organization  have  shown  a  total 
life  pattern  of  successful  endeavors.  They  were  good  in 
college,  are  active  in  taking  advantage  of  leadership 
opportunities,  and  see  themselves  as  forceful,  dominant, 
assertive,   and  confident"   (Campbell  et  al . ,  1970). 

Harrell  and  Harrell  (1974)  conducted  a  predictive  study 
of    MBA    graduates     in     order    to    determine    predictors  of 


management  success.  They  found  a  significant  relationship 
between  biographical  information  and  salary  level  attained  10 
years  later. 

Gantz,  Erickson,  and  Stephenson  (in  Miner,  1977)  conducted 
a  concurrent  study  of  research  scientists  and  engineers  for 
the  purpose  of  gaining  "more  theoretical  understanding  of  why 
some  persons  in  a  research  and  development  population  achieve 
promotion,  particularly  into  formally  designated  supervisory 
or  managerial  positions,  and  other  persons  do  not"  (p.  18)  . 
The  organization  was  a  federally  funded  public  laboratory. 
The  independent  variable,  the  selection  method  of  biographical 
information,  was  found  to  be  not  related  to  either  promotion 
rate  or  supervisor  ratings. 

Hinrichs  (1978)  found  a  significant  relationship  between 
biographical  information  and  administrative  level  achieved 
eight  years  later  for  a  sample  of  30  managers.  The  resulting 
correlation  coefficient  was  slightly  higher  than  that  for  the 
Overall  Assessment  Rating. 

Turnage  and  Muchinsky  (1984)  sought  to  examine  the 
predictability  of  assessment  center  evaluation  versus 
traditional  measures  in  forecasting  job  success.  As  part  of 
the  study,  they  separately  compared  biographical  information 
with  supervisor  ratings  and  salary  level  attained  up  to 
several  years  later.  These  variables  were  insignificantly 
related  for  the  mostly  nonmanagement  personnel  tested.  The 
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researchers  concluded  that  certain  background  data  predict 
criteria  as  well  as  assessment  center  data. 

Conclusions .  Compared  to  the  interview  method,  Levine  and 
Flory  (1975)  found  the  application  blank  information  to  be 
superior  in  accuracy  of  information.  However,  they  noted  that 
"sizable  inaccuracies"  had  been  found  in  each  method  (p.  384)  . 
Childs  and  Klimoski  (1986)  found  that  an  outgoing  and  self- 
confident  personality  preordains  success  in  both  job-  and  non- 
job-related  situation.  They  also  found  that  success  in  one's 
educational  history  may  aid  the  attainment  of  success  in  one's 
career,  regardless  of  whether  that  job  is  personally 
satisfying.  Most  studies  that  showed  significant  positive 
correlations  between  biographical  information  and 
administrative  achievement  attributed  the  relationship  to 
personality  characteristics  which  were  also  positive 
attributes  of  leadership  ability. 
Peer  Ratings  as  a  Selection  Method 

Peer  ratings  of  supervisory  potential  achievement  consist 
of  impressions  gained  from  interactions  in  an  egual,  non- 
supervisor-subordinate,  nature.  They  are  predictions  of  how 
well  a  peer  will  do  in  a  supervisory  position,  should  he  or 
she  be  placed  in  one   (Korman,   1968)  . 

Among  the  more  consistent  findings  concerning  peer  ratings 
is  the  significant  validity  these  afford  in  predicting  later 
performance.  Many  studies  conducted  in  military  settings 
indicated    that    peer    evaluations    during    officer  training 


successfully  predicted  later  criteria  of  performance. 
Williams  and  Leavitt  (1947)  found  that  peer  ratings  were 
better  predictors  of  long-term  success  in  the  Marine  Corps 
than  were  superiors'  ratings.  Much  military  research  was 
contributed  by  Hollander  (1954,  1956,  1957,  1964,  1965)  and 
the  evidence  was  consistently  favorable.  La  Gaipa  (1960) ,  in 
the  Development  of  the  Officer  Candidate  Biographical 
Information  Blank,  sampled  Naval  officers  in  both  shore  duty 
and  fleet  duty.  He  found  a  significant  correlation  between 
peer  ratings  and  later  performance  criterion. 

There  has  also  been  evidence  of  the  utility  of  peer 
ratings  from  other  spheres  of  activity,  as  Weitz  found  a 
relationship  of  r  =  .40  between  peer  nominations  and  later 
ratings  of  life  insurance  agents  in  a  supervisory  position 
(1958) .  As  Hollander  has  stated,  "when  employed  with 
discrimination,  it  [the  peer  rating  method]  can  provide  a 
unique  contribution  to  evaluation"   (1965,   p.   434) . 

Research  used  in  this  study.  A  total  of  8  research 
studies  were  found  which  used  peer  ratings  as  a  predictor 
variable  and  met  the  other  delimitations  of  this  dissertation. 
These  studies  yielded  10  correlation  coefficients  which  ranged 
in  strength  from  r  =  . 04  to  r  =  . 53 .  Samples  ranged  in  size 
from  40  to  799  persons. 

Haggerty  (1963) ,  in  two  separate  predictive  studies, 
explored  the  predictive  ability  of  peer  ratings  obtained  on 
Military  Academy  Cadets.      The   first  group  of  men  showed  a 


significant  correlation  with  the  outcome  variable  of 
supervisors'  ratings.  The  second  group  of  cadets  showed  a 
significant  correlation  with  a  later  combination  of  overall 
ratings . 

Roadman's  (1964)  predictive  study  compared  peer  ratings, 
taken  in  a  management  school  setting  at  IBM,  to  promotion 
rates  of  a  sample  of  graduates  who  had  received  promotions  at 
least  two  years  later.  A  significant  correlation  was  found. 
He  concluded  that  a  careful  and  comprehensive  peer  rating 
administered  in  a  middle  manager  training  program  can  identify 
those  who  later  move  into  senior  executive  positions. 

Hollander  (1965) ,  while  at  the  State  University  of  New 
York  at  Buffalo,  did  a  predictive  study  comparing  the 
relationship  between  peer  assessments  and  supervisor  ratings. 
The  study  was  begun  in  1955  at  the  Naval  Officer  Candidate 
School  (OCS)  in  Newport,  Rhode  Island.  An  entire  OCS  class 
was  made  available  for  this  investigation.  Four  forms,  each 
setting  out  different  gualities  to  be  rated,  were  utilized; 
these  dealt  with  leadership,  motivation  for  naval  service, 
probability  of  success  in  OCS  training,  and  success  as  a 
future  officer.  Results  of  part  of  the  study  were  reported 
by  Hollander  (1956)  and  showed  that  very  early  in  the  training 
program,  students  were  able  to  accurately  determine  which  of 
their  peers  would  do  well  after  training. 

In  the  follow-up  study  (Hollander,  1965) ,  supervisor 
ratings  of  former  trainees,  who  had  become  officers,  were  used 
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to  test  the  validity  of  peer  prediction  of  future  success. 
This  follow-up  of  officers  at  least  3  years  after  graduation 
showed  a  significant  correlation.  Hollander  concluded  that 
peer  nominations  used  early  in  training  can  make  a  distinctive 
contribution  to  the  prediction  of  a  long-range  criterion  of 
performance  after  training. 

Lawler  (1967) ,  while  working  for  the  Department  of 
Administrative  Sciences  of  Yale  University,  explored  the 
relationship  between  peer  ratings  and  supervisors'  ratings. 
In  this  concurrent  validity  study,  a  sample  of  mid-and  top- 
level  managers  in  a  manufacturing  company  were  rated  by 
several  peer  raters  on  three  traits.  Lawler  found  that 
personnel  selection  "decisions  (which  include  peer  ratings) 
will  be  of  a  higher  guality  than  if  just  superiors'  ratings 
are  relied  upon"   (p.   378) . 

Mayfield  (1970) ,  while  employed  with  the  Life  Insurance 
Agency  Management  Association,  studied  the  relationship 
between  peer  ratings  and  supervisors'  ratings  2.5  years  later. 
The  purpose  of  this  predictive  study  was  to  investigate  the 
value  of  the  buddy  nomination  procedure  in  the  selection  of 
assistant  managers  for  life  insurance  companies.  The  scores 
were  not  used  in  promotion  decisions  to  prevent  criterion 
contamination . 

Two  of  the  three  samples  showed  significant  correlations 
between  peer  and  supervisor  ratings.    Mayfield  concluded  that 


peer  ratings  can  be  made  in  a  "realistic"  administrative 
setting  and  retain  their  predictiveness. 

Mitchel  (1975) ,  while  at  Bowling  Green  State  University, 
studied  the  predictive  abilities  of  parts  of  the  Standard  Oil 
Company  of  Ohio's  assessment  center  program.  The  relationship 
between  peer  ratings  and  a  later  measure  of  salary  level 
attained  proved  significant.  He  concluded  that  "peer  and 
assessor  ratings,  as  well  as  combinations  of  variables,  were 
predictive  of  a  salary  criterion  of  managerial  success" 
(Mitchel,    1975,   p.  578). 

Gantz,  Erickson,  and  Stephenson  (in  Miner,  1977)  sought 
to  determine  why  some  persons  in  a  research  and  development 
population  achieve  promotion  into  formally  designated 
supervisory  or  managerial  positions  and  whether  those  who  were 
promoted  were  the  ones  who  should  have  been  promoted.  A  study 
of  research  scientists  and  engineers  in  a  large,  federally- 
funded  laboratory  showed  a  significant  correlation  between 
peer  ratings  and  promotion  rate.  These  peer  ratings  of 
overall  ability  and  perceived  creativity  were  statistically 
significant  at  the  .01  level. 

Turnage  and  Muchinsky  (1984),  examined  the  ability  of 
assessment  center  evaluations  to  predict  actual  job 
performance  criteria,  and  to  compare  the  predictability  of 
assessment  center  evaluations  versus  traditional  measures  in 
forecasting  job  success.  Their  data  came  from  a  one-day 
supervisory      selection     program      developed      by      a  large 
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manufacturing  firm.  The  outcome  data  were  gathered  from  1 
month  to  several  years  after  assessment  from  a  sample  of 
previous  assessees.  None  of  the  peer  ratings  was  significant. 
Several  possible  explanations  for  the  low  validity 
coefficients  were  given.  These  included  poor  experimental 
control,  insufficiently  trained  assessors,  aggregation  of  data 
over  a  4 -year  period,  and  the  effects  of  intervening 
variables . 

Conclusions .  There  appears  to  be  a  reasonable  basis  to 
contend  that  peer  nominations  do  provide  distinctive 
prediction  of  performance  at  a  considerably  later  time.  Peer 
evaluations  are  relevant  because  peers  are  situated  to 
evaluate  how  a  person  performs  in  terms  of  the  lateral 
relationships  in  working  toward  organizational  goals. 
Further,  peers  often  see  the  worker  at  times  when  his  superior 
is  not  viewing  his  behavior  and,  therefore,  they  may  see 
aspects  of  his  behavior  of  which  the  superior  is  not  aware 
(Lawler,  1967)  .  However,  attention  must  be  paid  to  the 
question  of  the  willingness  of  participants  to  rate  each  other 
accurately  if  they  know  full  use  will  be  made  of  peer  ratings 
for  administrative  purposes  (Roadman,  1964) .  A  problem  Lawler 
(1967)  acknowledged  was  that  if  the  peer  rater  knew  that  his 
or  her  opinion  was  "going  to  count"  in  an  employment  setting, 
ratings  may  lose  their  validity,  "particularly  if  a  situation 
exists  where  an  individual's  self-interests  might  be  best 
served  by  distortion  of  the  peer  ratings"  (p.  379) .  According 
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to    Korman    (1968)    there    is    a    need    for   the    initiation  of 
research  on  the  selection  method  of  peer  ratings  concerning 
both  its  predictive  validity  and  the  general  characteristics 
that  correlate  with  peer  ratings. 
Self- Appraisals  as  a  Selection  Method 

Self-ratings  of  administrative  ability  are  relevant  in 
selection  because  the  individual  has  more  information  about 
his  own  behavior  than  anyone  else  and  because  self-perceptions 
are  important  determinants  of  an  individual's  future  behavior 
(Lawler,  1967,  p.  371) .  A  negative  view  of  self-selection  for 
administrative  positions  has  apparently  been  shared  by  those 
behavioral  scientists  who  study,  develop,  and  validate 
personnel  selection  procedures.  The  reason  for  this  appears 
to  be  rooted  in  a  Theory  X  view  of  people  as  they  are  expected 
to  behave  in  a  personnel  selection  setting.  In  these  settings 
people  are  assumed  to  lack  objectivity  in  assessing  their  own 
performance  or  personal  attributes.  They  can  be  expected  to 
overestimate  their  performance,  skills,  knowledge,  and 
abilities  to  improve  their  chances  for  appointment  (Levine, 
1978)  . 

Published  studies  on  self-assessment  of  skills,  abilities, 
knowledge,  and  other  applicant  attributes  as  predictors  of 
administrative  job  performance  have  been  virtually  nonexistent 
until  recently  (Levine,  1978,  p.  230).  Studies  on  self- 
selection,  done  before  1962,  have  dealt  mainly  with 
objectively  measurable  traits   (Nickels  &  Renzaglia,   1958) . 
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Research  used  in  this  study.  A  total  of  8  research 
studies  were  found  which  used  self-appraisals  as  a  predictor 
variable  and  met  the  other  delimitations  of  this  dissertation. 
These  studies  yielded  9  correlation  coefficients  which  ranged 
in  strength  from  r  =  .01  to  r  =  .26.  Samples  ranged  in  size 
from  30  to  799  persons. 

Prien  and  Liske  (1962) ,  while  at  Case-Western  Reserve 
University,  explored  the  relationship  between  supervisor 
ratings  of  job  performance  and  incumbent  self-ratings  of  job 
performance  on  tasks  which  were  intangible  in  nature.  In 
their  concurrent  validity  study  of  employees  of  various 
corporations,  significant  correlations  were  found  between 
self -ratings  and  first-level  supervisor  ratings,  and 
insignificant  correlations  were  found  between  self-ratings  and 
second-level  supervisor  ratings.  The  preponderance  of  studies 
showed,  as  this  one  did,  that  individuals  rate  themselves 
higher  than  they  are  rated  by  comparison  groups. 

Lawler  (1967)  while  with  the  Department  of  Administrative 
Sciences  of  Yale  University,  studied  mid-  and  top-level 
managers  in  a  manufacturing  organization.  Four  coefficients 
of  the  relationship  between  self -appraisal  and  supervisory 
job  performance  ratings  showed  an  average  nonsignificant 
correlation.  This  concurrent  validity  study  also  showed 
evidence  of  self  over-estimation  of  perceived  administrative 
abilities. 
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Thornton  (1968) ,  while  a  summer  associate  at  the  firm  of 
Rohrer,  Hibler  and  Replogle  in  Chicago,  investigated  the 
relationship  between  supervisory  perceptions  and  incumbent 
self-perceptions  of  the  performance  of  executive  personnel. 
A  sample  of  high-level  managers  in  a  large  corporation  were 
included  in  this  study.  An  average  of  coefficients  of  27 
traits,  or  behavioral  characteristics,  studied  showed  a 
significant  correlation  between  self -ratings  and  supervisor 
perceptions  of  performance  ratings.  The  tendency  was  for 
self-evaluations  of  performance  to  be  higher  than  supervisory 
perception  and,  in  this  study,  those  executives  who  overrated 
themselves  were  considered  least  promotable. 

Campbell  et  al.  (1970),  in  a  long-term  staff  study, 
reviewed  the  relationship  between  self-assessment  and  a 
combination  of  performance  ratings  to  include  salary, 
supervisory  ratings,  and  administrative  level  attained.  This 
was  part  of  the  Early  Identification  of  Management  Potential 
(EIMP)  study  carried  out  by  the  Standard  Oil  Company  of  New 
Jersey  (SONJ) .  A  sample  of  managers  of  SONJ  was  used.  The 
resulting  correlation  coefficient  was  significant. 

Contrary  to  previous  evidence,  Heneman  (1974)  found  a 
tendency  for  self -ratings  to  be  less  lenient  than  supervisory 
ratings,  with  a  significant  correlation.  His  concurrent  study 
compared  self-  and  supervisor  ratings  of  job  performance  of 
former  MBA  graduates  of  Indiana  University,  7  to  10  years 
after    graduation.       He    suggested    that    future    research  on 
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managerial  performance  should  include  self -ratings  where  it 
is  made  clear  that  these  ratings  will  be  used  for  research 
purposes  only. 

Hinrichs  (1978)  completed  one  of  the  few  predictive 
validity  studies  of  self-assessment  of  administrative 
performance.  The  predictions  of  the  assessment  center  were 
not  used  administratively  to  prevent  criterion  contamination. 
This  study  is  included  in  an  8-year  followup  of  a  management 
assessment  center  at  International  Business  Machines  (IBM) . 
Self-ratings  of  marketing  personnel  correlated  significantly 
with  the  outcome  variable  of  administrative  level  achieved. 

Steel  and  Ovalle  (1984) ,  while  at  the  Air  Force  Institute 
of  Technology,  sought  to  compare  the  relative  validity  of 
self-ratings  for  predicting  objective  criteria  of  managerial 
job  performance.  A  concurrent  validity  study  at  a  large 
lending  institution  revealed  an  insignificant  correlation 
between  self-rating  and  supervisor  rating. 

Turnage  and  Muchinsky  (1984)  conducted  a  predictive  study 
in  a  large  manufacturing  firm.  The  subjects  were  employees 
who  were  subseguently  promoted  to  supervisory  positions. 
Self-evaluations  of  job-related  personal  characteristics,  such 
as  ability  to  withstand  stress,  intellectual  abilities,  and 
interpersonal  skill,  made  up  the  independent  variable.  Self- 
evaluations  related  insignificantly  and  negatively  with  both 
the  outcome  variables  of  salary  level  attained  and  supervisor 
ratings.     The  problem  of  criterion  contamination   (i.e.,  the 
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availability  of  assessment  scores  to  superiors)  may  have  been 
a  factor  in  this  study. 

Conclusions .  It  has  been  suggested  by  Levine  (1978)  that, 
as  predictors  of  job  performance,  self -assessments  should  at 
least  supplement  other  information.  Self-assessments  may 
replace  more  traditional  selection  methods,  especially  in 
measuring  psychological  attributes  that  may  be  relatively 
inaccessible  by  other  means.  He  suggested  that  self- 
assessments  might  have  a  motivational  impact  on  those 
applicants  who  are  hired,  as  these  people  will  strive  to  be 
consistent  with  their  self-perceived  competencies.  Levine 
admits  that  research  is  needed  to  determine  what  applicant 
attributes  are  most  validly,  or  least  validly,  self-assessed. 
Assessment  Center  Processes  as  a  Selection  Method 

A  highlighting  feature  of  the  assessment  center  is  that 
candidates  are  evaluated  not  on  what  they  have  done  in  present 
or  past  jobs  but  on  how  they  are  likely  to  cope  with  a  new 
type  of  position.  This  involves  using  various  situational 
tests  as  well  as  incorporating  some  of  the  more  classic 
selection  procedures,  such  as  aptitude  tests  and  interviews. 
Assessments  are  conducted  at  least  partially  in  groups,  which 
permits  observing  group  interactions  as  well  as  obtaining  peer 
ratings   (Howard,   1974) . 

The  following  dimensions  are  most  often  assessed:  (a) 
leadership,  (b)  organizing  and  planning,  (c)  decision  making, 
(d)   oral  and  written  communications  skills,    (e)  initiative, 
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(f)  energy,  (g)  analytical  ability,  (h)  resistance  to  stress, 
(i)  use  of  delegation,  (j)  behavior  flexibility,  (k)  human 
relations  competence,  (1)  originality,  (m)  controlling,  (n) 
self-direction,  and  (o)  overall  potential   (Howard,   1974) . 

The  origin  of  the  use  of  multiple  assessment  procedures 
on  a  large  scale  is  credited  to  German  military  psychologists. 
The  British  adapted  the  procedures  to  the  screening  of  officer 
candidates,  and  the  United  States  Office  of  Strategic  Services 
(OSS)  took  over  the  approach  from  the  British  during  World  War 
II  (Bray  &  Grant,  1966) .  Howard  (1974)  stated  that  the  first 
industrial  use  of  an  assessment  center  has  been  generally 
attributed  to  AT&T  beginning  in  1956. 

Research  used  in  this  study.  A  total  of  14  research 
studies  were  found  that  used  Overall  Assessment  Ratings  (OAR) 
of  an  assessment  center  process  as  a  predictor  variable  and 
met  the  other  delimitations  of  this  dissertation.  These 
studies  yielded  16  correlation  coefficients  which  ranged  in 
strength  from  r  =  .  00  to  r  =  .  65 .  Samples  ranged  in  size  from 
25  to  5,943  persons. 

The  first  industrial  use  of  an  assessment  center  is 
generally  attributed  to  the  American  Telephone  and  Telegraph 
Company.  Other  centers  have  been  variations  on  AT&T's  theme. 
The  AT&T  experimentation  was  research-oriented  and  designed 
to  follow  the  development  of  managerial  personnel  for  many 
years  after  assessment   (Howard,   1974) . 


87 

Bray  and  Grant  (1966)  measured  personal  characteristics 
hypothesized  to  be  of  importance  either  in  developmental 
change  in  early  adulthood  or  success  in  business  management. 
The  overall  assessment  rating  was  significantly  correlated 
with  both  salary  level  attained  and  administrative  level 
achieved  8  years  after  assessment.  This  was  one  of  few 
studies  in  which  criterion  contamination  was  eliminated.  The 
researchers  concluded  that  "no  single  characteristic 
determines  progress  in  management"   (Bray  &  Grant,   1966) . 

Moses  (1972)  studied  the  relationship  between  assessment 
center  processes  and  managerial  achievement.  Using  a  large 
sample  of  assessees  of  an  AT&T  one-day  assessment  center, 
Moses  found  a  significant  relationship  between  overall 
assessment  rating  and  administrative  level  achieved. 

Moses  and  Boehm  (1975)  studied  the  relationship  between 
the  assessment  center  process  and  subseguent  progress  in 
management  for  women  in  the  Bell  System.  Assessment  center 
data  were  obtained  on  4,84  6  nonmanagement  women  from  1963 
until  1971.  The  conclusion  the  researchers  drew  in  this  study 
was  that  the  assessment  center  process  predicted  the  future 
performance  of  women  as  accurately  as  it  did  that  of  men. 

Huck  and  Bray  (1976)  tested  the  validity  of  an  assessment 
center  process  on  a  population  different  from  the  population 
of  the  original  Management  Progress  Study.  Using  a  relatively 
small  sample  of  females  who  had  been  assessed  previously,  the 
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researchers  found  a  significant  correlation  between  overall 
assessment  rating  and  later  administrative  achievement. 

The  International  Business  Machines  (IBM)  Corporation 
assessment  center  was  patterned  after  the  AT&T  model  and 
included  2.5  days  of  assessment  activity  followed  by  2  days 
of  developmental  activity.  Several  studies  have  reported 
results  of  the  IBM  assessment  center  (Hinrichs,  1978;  Kraut 
&  Scott,  1972;  Roadman,  1964;  and  Wollowick  &  McNamara,  1969). 

Wollowick  and  McNamara  (19  69)  studied  the  progress  of  low- 
and  mid-management  males  as  subjects.  The  purpose  of  the 
study  was  to  determine  the  validity  of  an  assessment  center 
approach  in  predicting  management  potential  and  to  determine 
the  relative  value  of  each  of  the  assessment  center 
components.  An  overall  assessment  rating  was  determined  for 
each  subject  at  the  conclusion  of  the  two-day  assessment 
session.  The  four  observers  who  formed  the  assessment  staff, 
and  who  were  operating  management  personnel  at  least  two 
levels  above  the  participants,  assigned  each  subject  an 
overall  assessment  rating  taking  into  consideration  all  of  the 
variables  in  the  program.  According  to  the  overall  assessment 
rating,  each  subject  was  ranked  on  a  5-point  scale  rating 
potential  for  advancement  within  the  company  (Wollowick  & 
McNamara,   1969) . 

These  overall  assessment  ratings  were  compared,  2.5  to  3.2 
years  later,  to  the  subjects'  administrative  levels  achieved. 
This  resulted  in  a  significant  correlation.     It  was  evident 
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to  the  researchers  that  the  subjectively  derived  overall 
assessment  rating  utilized  in  this  assessment  program  was  a 
valid  predictor  of  management  success. 

Kraut  and  Scott  (1972)  studied  the  data  provided  by  the 
IBM  assessment  center  on  the  progress  of  non-management  males 
into  administrative  positions.  The  purpose  of  the  study  was 
to  examine  the  validity  of  the  assessment  program  in 
predicting  administrative  achievement.  Overall  assessment 
ratings  were  correlated  with  administrative  level  achieved 
from  1  to  5  years  later  using  the  same  5-point  rating  system 
that  was  used  in  the  Wollowick  and  McNamara  (1969)  study.  The 
correlation  between  overall  assessment  rating  and  later 
achievement  was  significant.  This  assessment  program  was  one 
of  the  few  selection  methods  which  showed  validity  in 
predicting  success  beyond  the  first-level  promotion  after 
assessment. 

Hinrichs  (1978)  completed  a  predictive  study  using  a  small 
sample  of  IBM  managers.  A  significant  relationship  was  found 
between  assessment  center  overall  assessement  ratings  and 
administrative  level  achieved  eight  years  later.  However, 
Hinrichs  also  found  that  the  relatively  simple  prediction 
based  upon  managerial  review  of  the  personnel  files  did  as 
well  in  prediction  as  the  assessment  center. 

Researchers  in  the  IBM  studies  reviewed  here  concluded 
that  large-scale  assessment  programs  appear  useful  in  making 
discriminations    of    management    potential    which    are  later 
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confirmed  by  the  rate  of  promotions,  as  well  as  demotions 
(Kraut  &  Scott,  1972) .  The  studies  contained  criterion 
contamination  as  the  subjects'  results  were  provided  to  their 
supervisors.  However,  it  was  observed  that  the  "relationship 
of  ratings  to  first  promotions  is  moderate  enough  to  reduce 
fears  of  'crown  prince'  or  'kiss  of  death'  effects"  (p.  124). 
According  to  Kraut  &  Scott   (1972) , 

compared  to  the  normal  promotional  system  in  most 
companies,  the  program  typically  has  some  obvious 
advantages  in  reliability  and  validly  measuring 
management  potential.  Instead  of  judgments  by  one's 
immediate  manager  which  may  be  more  or  less 
subjective,  evaluations  in  the  program  are  made  by 
several  managers  (raters)  who  are  likely  to  be  much 
more  objective.  Further,  they  are  not  making 
judgments  about  the  individual's  management  potential 
from  his  performance  in  a  non-management  job;  instead 
they  evaluate  all  candidates  against  a  common 
yardstick  comprised  of  standardized  management-type 
tasks.     (p. 124) 

The  Standard  Oil  Company,  Ohio  (SOHIO) ,  created  an 
assessment  center  in  1963.  This  Formal  Assessment  of 
Corporate  Talents  (FACT)  Program  was  modeled  after  the  AT&T 
assessment  process  (Carleton,  1970) .  Most  data  from  the  SOHIO 
program  pointed  to  the  superiority  of  the  overall  assessment 
ratings  but  did  not  eliminate  any  single  category  of 
assessment  components   (Howard,   1974) . 

Carleton  (1970)  sought  to  determine  the  relationships 
between  test  and  rating  data  and  later  measures  of 
administrative  achievement.  A  significant  correlation  was 
found  between  the  overall  assessment  rating  and  subsequent 
supervisor  ratings.      Carleton   concluded  that  both  test  and 
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rating  data  generated  in  a  multiple  technique  assessment 
center  held  up  well  as  predictors  of  behavioral  ratings  made 
several  years  later. 

Finley  (1970)  studied  the  predictive  validity  of 
projective  tests  in  the  SOHIO  management  assessment  center. 
Using  a  sample  derived  from  the  FACT  Program,  he  found  a 
significant  relationship  between  overall  assessment  ratings 
and  subsequent  supervisor  ratings. 

Mitchel  (1975)  studied  a  sample  of  managers  from  SOHIO 1 s 
FACT  Program.  An  insignificant  correlation  was  found  between 
overall  assessment  ratings,  minus  assessor's  ratings  of 
potential,  and  salary  level  attained  5  years  after  assessment. 
The  overall  rating  was  not  the  most  valid  predictor  in  this 
study,  nor  did  it  appear  in  any  of  the  stepwise  equations  done 
by  the  researcher. 

The  SOHIO  studies  involved  both  trained  and  untrained 
assessors.  Evidence  for  discriminant  validity  was  found  for 
the  two  trained  assessor  groups,  but  not  for  the  group  in 
which  assessors  consisted  of  untrained  supervisors.  It  was 
concluded  that  the  assessment  center  method  was  able  to 
predict  multiple  criteria  fairly  well,  but  that  the  poor 
quality  of  criterion  measures  probably  reduced  the  convergent 
validity  coefficients   (Howard,   1974) . 

One  of  the  most  extensive  validations  of  the  assessment 
center  method  against  measures  of  management  effectiveness  was 
the  Early  Identification  of  Management  Potential  (EIMP)  study 
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carried  out  by  the  Standard  Oil  Company  of  New  Jersey  (SONJ) . 
The  long-term  staff  study  was  extensively  reported  in  Campbell 
et  al.    (1970) . 

Through  this  study,  the  management  of  SONJ  sought  to 
determine  how  employees  who  possess  the  potential  to  be 
successful  in  management  can  be  identified  early  in  their 
careers.  Management  personnel  of  SONJ  were  more  hopeful  of 
identifying  young  men  with  potential  for  general  management 
positions  and  were  less  concerned  with  pinpointing  those  with 
potential  narrowly  oriented  toward  specific  functional  areas. 
A  significant  relationship  was  found  between  overall 
assessment  ratings  and  a  combination  of  outcome  variables  in 
the  concurrent  study. 

Schmitt,  Noe,  Meritt,  and  Fitzgerald  (1984)  sought  to 
evaluate  the  use  of  the  assessment  center  approach  when 
applied  in  an  educational  setting  to  select  secondary  and 
elementary  school  administrators.  Using  an  assessment  center 
designed  by  the  National  Association  of  Secondary  School 
Principals  (NASSP)  and  the  American  Psychological  Association, 
they  found  a  significant  relationship  between  overall 
assessment  rating  and  subseguent  supervisor  ratings.  However, 
the  number  of  years  between  the  measurement  of  variables  was 
limited  due  to  the  fact  that  average  job  tenure  of  the 
promoted  participants  was  only  13  months.  This  study 
represented  the  first  validation  of  the  use  of  the  assessment 
center  process  in  the  selection  of  school  administrators. 
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Turnage  and  Muchinsky  (1984)  conducted  a  study  based  upon 
data  from  a  one-day  assessment  center  developed  by  an 
unidentified  large  manufacturing  firm.  The  subjects  were  799 
nonmanagement  employees  who  were  subsequently  promoted  to 
supervisory  positions.  Two  overall  assessment  ratings  were 
provided  using  two  separate  outcome  criteria,  salary  level 
attained  and  supervisor  ratings.  Based  on  these,  and  other 
findings  in  the  study,  the  researchers  concluded  that  there 
appeared  to  be  no  appreciable  relationship  between  how  one  is 
evaluated  in  an  assessment  center  and  how  one  performs  on  the 
job.  They  stated  that  "there  is  not  a  strong  relationship 
between  job  performance  and  promotion"   (p.   600) . 

A  number  of  reasons  were  surmised  by  the  researchers  for 
the  lack  of  correlation.  Methodological  factors  such  as  low 
criterion  reliability,  low  predictor  reliability,  severe 
restriction  of  range,  marked  skew  in  the  data,  procedural 
inconsistencies,  lack  of  comparability  across  assessment 
groups,  and  errors  in  data  collection  were  cited  as  problems 
within  this  study  (Turnage  &  Muchinsky,  1984). 

Rankin  (1981)  conducted  a  predictive  validity  study  in  an 
assessment  center  program  in  a  military  organization. 
Subjects  were  low-level  management  civilian  employees.  No 
relationship  was  found  between  overall  assessment  ratings  and 
a  combination  of  outcome  measures.  In  fact,  Rankin  (1981) 
stated  that, 

the  most  significant  correlation  between  overall 
ratings  was  found  within  the  group  of  assessees  with 
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military  supervisors  (N  =  19)  .  In  a  comparison  of 
OVERAC  (OAR) ,  the  overall  1981  rating,  a  correlation 
of  -  .40  was  achieved,  representing  a  significance  of 
.045.  This  could  be  considered  to  be  a  good  result 
were  it  not  for  the  fact  that  it  is  a  negative 
coefficient.  This  implies  that  military  supervisors 
rate  their  personnel  opposite  from  that  of  the 
assessment  center.  For  example,  if  the  assessment 
center  rated  an  individual  poorly,  these  results 
indicate  that  chances  are  that  this  individual's 
military  supervisor  would  rate  him  highly.      (p.  67) 

Conclusions .      In  most  cases,    the  ratings  of  candidates 

based  on  the  totality  of  assessment  procedures  seemed  to  have 

validity  superior  to  any  of  the  specific  components.  Since 

the  situational  tests  represented  a  unique  contribution  to  the 

process,  and  were  relied  upon  heavily,   it  was  usually  assumed 

that  these  were  what  made  the  difference,   although  specific 

data  often  were  not  reported.     The  in-basket's  contribution 

usually  was  considered  critical,    although   it  mainly  tapped 

administrative  skills  and  was  thus  more  narrow  in  scope  than 

some  of  the  other  exercises.     Other  situational  tests  were  so 

varied,    and    data    were    so    seldom    reported    on    them,  that 

conclusions  were  difficult  to  draw.    The  mental  ability  tests 

seemed    to    work    for    some    companies    but    not    for  others; 

personality  tests  showed  moderate  to  little  success,  but  they 

continued  to  be  used.      The  projectives  and  interviews  were 

more   successful   than   expected   from  their  past  reputations. 

More  research  evidently  is  needed  on  the  various  components 

of  assessment  centers  and  their  integration  (Howard,   1974) . 
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Summary  and  Critique 

The  job  of  the  administrator  has  changed  considerably. 
It  is  perceived  as  more  complex  than  ever  before,  requiring 
multiple  skills  and  abilities,  and  is  affected  by  many  outside 
forces.  Some  selection  methods  of  administrative  personnel 
are  used  in  sequence  as  screening  devices,  whereas  other 
methods  consider  many  different  personal  qualities  in  concert 
in  the  evaluation  of  a  job  candidate. 

The  use  of  one  or  several  selection  methods  is  a  function 
of  the  importance  of  the  job,  the  money  and  the  time  available 
for  selection,  and  the  accuracy  of  any  one  method  compared 
with  a  combination  of  methods.  Research  studies  can  be  cited 
that  arrive  at  opposite  conclusions.  In  some  cases,  one 
selection  method  can  be  superior  to  several  methods.  In  other 
cases  a  combination  of  methods  is  superior  to  any  one. 
Theoretically,  several  methods  can  never  be  inferior  to  one 
if  they  are  combined  on  a  statistically  sound  basis,  but  this 
is  rarely  done.  A  valid  method  can  be  reduced  in 
effectiveness  by  the  addition  of  an  inferior  one,  except  when 
the  less  valid  method  measures  attributes  not  otherwise 
detected  (Mandell,   1964,  p.  22). 

The  quality  of  validity,  addressed  in  this  study,  has  been 
studied  in  personnel  selection  research  and  literature  for 
decades.  Often  a  method  was  "sold"  based  upon  what  it  was 
claimed  to  measure,  without  any  evidence  that  the  claims  were 
justified.     In  some  cases,   the  endorsement  of  a  user  who  had 
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conducted  no  research  studies  was  used.  More  technical  means 
of  distortion  occurred  when  claims  of  overall  validity  were 
made  on  the  basis  of  the  outcome  scores  in  a  training  course, 
correlation  with  another  selection  method,  or  with  a  criterion 
that  represented  only  a  portion  of  the  total  job  requirements 
(Mandell,   1964,  p.   2) . 

As  Stahl   (1976)  stated: 

The  worst  sins  in  examining  personnel  are  not  inherent 
in  any  single  selection  instrument  or  method,  but  are 
derived  from  using  the  wrong  selection  instrument  or 
method,  using  something  that  is  inappropriate  for 
gauging  the  abilities  really  needed  in  a  given 
occupation  or  position.  When  the  wrong  measuring 
instrument  or  method  has  been  used,  too  often  the 
resulting  criticism  has  been  directed  against  the 
method  itself  instead  of  against  the  bad  judgment  of 
the  selectors.      (pp.  134-135) 

Previous  reviews  of  administrative  personnel  selection 
research  have  been  subjective  in  nature  (Howard,  1974;  Korman, 
1968) .  In  the  present  study,  the  statistical  synthesis  of  the 
research  data  has  been  done  using  the  objective  process  of 
meta-analysis.  Specific  procedures  used  in  the  conduct  of  the 
meta-analysis  of  administrative  personnel  selection  methods 


are  introduced  in  the  following  chapter. 


CHAPTER  III 
PROCEDURES 


Introduction 

In  this  chapter,  the  purpose  of  the  study,  the  rationale 
for  the  use  of  the  meta-analytic  approach,  the  application  to 
selection  research  for  administrative  personnel,  the 
identification  of  the  research  studies  used,  the 
classification  of  variables,  and  procedures  for  the  meta- 
analysis are  discussed.  These  procedures  include:  conversion 
to  a  common  metric;  averaging  correlations  across  studies  and 
variables;  elimination  of  sampling  error;  and  adjustments  for 
error  of  measurement. 

The  Purpose  of  the  Study 

The  purpose  of  this  study  was  to  develop  a  set  of 
guidelines  for  improving  the  selection  of  administrators  of 
public  community  colleges.  Toward  this  end,  the  questions 
posed  in  the  Statement  of  the  Problem  in  Chapter  I  have  been 
defined  here  in  greater  detail.  In  Chapter  IV,  the 
presentation  of  the  results  of  the  meta-analysis,  the 
following  questions  are  answered. 

1.  What  methods  for  the  selection  of  administrative 
personnel      are     commonly     used     by     corporations,  public 
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administration  agencies,  the  military  services,  and 
institutions  of  higher  education? 

a.  What  are  the  most  commonly  used  administrative 
selection  methods  across  all  types  of  organizations? 

b.  What  are  the  most  commonly  used  administrative 
selection  methods  in  each  type  of  organization? 

c.  How  does  the  frequency  of  use  of  each  selection 
method  relate  to  its  ability  to  predict  administrative 
achievement? 

2 .  What  does  research  demonstrate  are  the  methods  which  are 
significantly  related  to  achievement  in  administrative 
positions? 

a.  To  what  extent  do  all  selection  methods,  taken 
inclusively,  correlate  with  administrative  achievement? 

b.  To  what  extent  do  different  selection  methods 
correlate  with  administrative  achievement? 

c.  To  what  extent  do  selection  methods,  taken  both 
inclusively  and  separately,  correlate  with  different  measures 
of  administrative  achievement? 

d.  To  what  extent  do  selection  methods  predict 
achievement  in  successively  higher  levels  of  administration? 

e.  To  what  extent  do  selection  results  of  the  assessment 
center  approach  correlate  with  administrative  achievement? 

f.  To  what  extent  does  the  type  of  interaction  (i.e., 
written,  verbal,  or  performance)   required  by  an  applicant  in 
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completing  a  selection  test  correlate  with  administrative 
achievement? 

3.  What  information,  derived  from  a  meta-analysis  of  the 
methodologies  of  selection  research,  is  useful  in  interpreting 
the  findings  of  this  study? 

a.  To  what  extent  does  criterion  contamination  effect 
the  correlation  between  selection  method  results  and 
administrative  achievement? 

b.  To  what  extent  does  restriction  of  range  effect  the 
correlation  between  selection  method  results  and 
administrative  achievement? 

4.  What  guidelines  can  be  derived  from  this  research  for 
improving  personnel  selection  methods  for  administrators  of 
public  community  colleges? 

a.  To  what  extent  do  the  methods  now  used  for  the 
selection  of  administrative  personnel  in  public  community 
colleges  correspond  with  those  methods  which  show  the  highest 
correlation  to  administrative  achievement? 

b.  What  selection  methods,  indicated  by  the  results  of 
the  meta-analysis,  could  be  added  to  or  dropped  from  the 
methods  currently  in  use  to  make  the  selection  process  more 
accurate  and  efficient? 

The  Rationale  for  the  Meta-analytic  Approach 

In  his  presidential  address  to  the  1976  meeting  of  the 
American  Educational  Research  Association  (AERA)  Glass  (1976) 
introduced  the  concept  of  meta-analysis.      In  distinguishing 
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meta-analysis  from  primary  analysis  (the  original  analysis  of 

data    from    a    research    study)     and    secondary    analysis  (the 

reanalysis  of  existing  data  in  order  to  answer  the  original 

research  question) ,  Glass  remarked: 

My  major  interest  currently  is  what  we  have  come  to 
call — not  for  want  of  a  less  pretentious  name — the 
meta-analysis  of  research.  The  term  is  a  bit 
grand,  but  it  is  precise,  and  apt. . . .  Meta-analysis 
refers  to  the  analysis  of  analyses.  I  use  it  to 
refer  to  the  statistical  analysis  of  a  large 
collection  of  analyses  results  from  individual 
studies  for  the  purpose  of  integrating  the 
findings.  It  connotes  a  rigorous  alternative  to 
the  casual,  narrative  discussions  of  research 
studies  which  typify  our  attempts  to  make  sense  of 
the  rapidly  expanding  research  literature.      (p.  3) 

The  major  thrust  of  meta-analysis  is  the  understandable 

integration     of     existing     research.         In     many     areas  of 

sociological,    psychological,    and   educational   research,  the 

need  to  understand  the  research  which  has  been  done  is  much 

greater  than  that  of  adding  another  study  to  the  storehouse 

of  knowledge.     This  situation  has  led  to  conflicting  results 

which  lead  to  no  acceptable  answers  to  the  problems  posed, 

but    instead    yields    unending    calls    for    further  research. 

Scholars  have  access  to  thousands  of  research  studies,  but 

progress  in  making  sense  out  of  them  for  any  given  topic  is 

painfully  slow.      Often  we   "know"  more  than  we  are  able  to 

"understand."     Furthermore,    literature  reviews  of  empirical 

studies     are     notorious     for    depending     on    the  subjective 

judgments,    preferences,    and    biases    of    the    reviewers,  and 
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conflicting  interpretations  of  the  evidence  are  not  uncommon 
(Wolf,   1986,  p.   10) . 

The  meta-analytic  method  permits  quantitative  reviews 
and  syntheses  of  the  research  literature.  "What  is  needed 
are  methods  that  will  integrate  results  from  existing  studies 
to  reveal  patterns  of  relatively  invariant  underlying 
relations  and  causalities,  the  establishment  of  which  will 
constitute  general  principles  and  cumulative  knowledge" 
(Hunter  et  al .  ,  1982,  p.  26).  In  the  meta-analytic  approach, 
"the  findings  of  multiple  studies  should  be  regarded  as  a 
complex  data  set,  no  more  comprehensible  without  statistical 
analysis  than  would  hundreds  of  data  points  in  one  study" 
(Glass  et  al.,   1981,  p.  12). 

The  Application  to  Administrative  Personnel  Selection  Research 
The  meta-analytic  approach  is  warranted  in  the  case  of 
administrative  personnel  selection  research.  Wolf  (1986) 
suggested  that  the  method  for  synthesizing  the  results  of 
correlational  studies  was  to  take  the  average  of  the 
correlations  between  the  two  variables  that  examine  the  same 
research  question  across  separate  research  studies.  According 
to  Hunter,  Schmidt,  and  Jackson  (1982) ,  "cumulation  of  results 
can  be  used  whenever  there  are  at  least  two  studies  with  data 
bearing  on  the  same  relation"  (p.  28)  .  They  specifically 
addressed  the  use  of  meta-analysis  in  synthesizing  personnel 
selection  research.  They  recognized  that  in  personnel 
selection  research  different  tests  were  often  used  to  measure 
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the  same  variable.  Thus,  "variation  in  reliability 
contributes  to  the  variation  in  correlations  across  studies 
if  the  review  covers  all  studies  using  a  given  predictor 
rather  than  a  fixed  test"   (p.  91) . 

Although  hundreds  of  studies  exist  and  many  claims  are 
made,  the  results  of  existing  administrative  personnel 
selection  research  are  unsettlingly  discrepant.  Fifty-two 
studies  met  the  fairly  strict  delimitations  set  forth  in  this 
research.  Several  studies  yielded  more  than  one  correlation 
coefficient.  A  total  of  128  correlation  coefficients  were 
derived  from  the  52  studies.  Care  was  taken  to  maintain 
independence  of  samples  within  and  between  studies.  Where 
subsamples  were  analyzed  within  a  study,  and  the  relationship 
between  the  same  variables  was  tested  in  each  subsample,  these 
subsamples  were  combined  to  yield  an  average  correlation 
coefficient.  The  few  negative  correlations  found  were  treated 
as  positive  for  the  reason  that  this  study  deals  with  the 
strengths,  not  the  directions,  of  the  methods  to  predict 
achievement.  The  strengths  of  correlations  between  selection 
methods  and  administrative  achievement  in  the  research  studies 
reviewed  ranged  from  r  =  .00  to  r  =  .65.  Their  mean 
correlation  was  r  =  .22  which  increased  to  r  =  .25  when  the 
correlations  were  weighted  for  sample  size  according  to 
formula  (3)   in  this  chapter. 

The  intent  of  meta-analysis  is  to  help  understand  what 
causes  variance  in  previous  studies  and  to  provide  a  much  more 


stable  framework  upon  which  tentative  theories  can  be  based. 
Meta-analysis  gives  the  researcher  the  capability  to  study  not 
only  the  subject  at  hand  but  also  the  frailties  of  the 
research  designs.  If  design  seems  to  be  a  problem  in  a  number 
of  studies,  or  if  specific  design  flaws  are  occurring 
regularly,  the  proper  approach  is  to  code  that  characteristic 
along  with  the  studies'  other  characteristics  and  empirically 
determine  how  much  of  the  variance  in  results  can  be  accounted 
for  by  such  design  and  analysis  flaws.  Such  coding  was  done 
in  this  research. 

The  rationale  and  need  for  meta-analysis  of  research 
findings  can  be  summarized  by  a  final  comment  from  Glass's 
AERA  speech: 

We  need  more  scholarly  effort  concentrated  on  the 
problem  of  finding  the  knowledge  that  lies  untapped 
in  completed  research  studies.     (p.  5) 

Identification  of  the  Research  Studies 

Many  research  studies  concerning  administrative  personnel 

selection    were    present    in    the    literature.        The  sources 

included      only      published      studies      found      in  journals, 

periodicals,  information  retrieval  systems,  and  books.  Glass 

(1976)  was  aware  that  information  was  plentiful  when  he  stated 

that: 

We  are  inundated  with  information.  The  ERIC  system 
fills  over  two  million  document  reguests  yearly. 
Some  have  termed  our  predicament  "the  misinformation 
explosion."  I  assess  it  differently;  we  face  an 
abundance  of  information.  Our  problem  is  to  find 
the  knowledge  in  the  information.      (p.  4) 
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Data  sources  concerning  the  selection  of  administrative 
personnel  for  the  period  January  1,  1962  through  December  31, 
1985  were  located  in  manual  and  computerized  indexes.  Books, 
periodicals,  and  microfiche  collections  of  The  Library  of 
Congress  and  libraries  of  the  University  of  Florida,  the 
University  of  South  Florida,  George  Mason  University,  the 
American  Psychological  Association,  and  various  Department  of 
Defense  installations  were  used.  Computerized  index  searches 
of  the  NEXIS,  Educational  Resources  Information  Center  (ERIC) , 
PSYCINFO,  the  Defense  Technical  Information  Center,  and  LC 
MARC  databases  were  obtained.  Research  studies  were  most 
often  found  published  in  professional  journals  in  the  fields 
of  psychology,  personnel,  public  administration,  and 
management. 

Classification  of  Variables 

Independent  variables.  This  research  dealt  with 
personnel  selection  methods  rather  than  with  individual 
selection  tests.  Selection  methods  were  used  as  the 
independent,  or  predictor,  variables  in  the  present  study. 
When  specific  tests  were  administered  in  a  research  study, 
each  test  was  recorded  by  the  type  of  selection  method  it 
represented.  In  a  1968  review  of  personnel  selection 
research,  Korman  delineated  selection  methods  in  the  following 
manner : 

I .     Psychometric  Prediction 
A.     Cognitive  Ability  Test 
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1.  verbal  ability 

2 .  mechanical  aptitude 

3.  adaptability 

4.  critical  thinking 

5.  clerical  ability 

B.  Objective  Personality  and  Interest  Inventory 

1.  preferences 

2.  vocational  interest 

3.  personality  inventory 

4 .  temperament  inventory 

5.  cooperation  test 

6.  risk  scale 

C.  Leadership  Ability  Test 

1.  leadership  opinion 

2.  sentence  completion 

3.  practical  judgment 

D.  Personal  History  Data 

1.  personal  history  blank 

2.  age,  education 

3 .  academic  grades 
II.     Judgmental  Prediction 

A.  Executive  Assessments   (assessment  centers) 

B.  Peer  Rating 

1.  overall  rating 

2.  critical  incident 

3 .  combat  rating 
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C.     Superior  and  Faculty  Rating 
A  look  at  the  research  studies  Korman   (1968)  reviewed 


revealed 

13  selection  methods  that  were  most  commonly  used. 

Of  these 

,  nine  appeared  to  be  mutually  exclusive  and  have  been 

used  as 

classifications  of  the  independent  variables  for  the 

purpose 

of  this  study: 

1 . 

aptitude  and  intelligence  measures, 

2  . 

personal  interviews, 

3  . 

job-related  skills  indicators, 

4  . 

psychological  attribute  indicators, 

5. 

values  systems  assessments, 

6. 

biographical  information, 

7  . 

peer  ratings, 

8  . 

self-appraisals,  and 

9. 

assessment  center  processes. 

For  the  purpose  of  this  study,  the  numerous  correlations 

contained  in  each  report  of  an  assessment  center  process  have 
been  separated  and  used  independently,  by  selection  method, 
where  possible.  In  addition,  the  overall  assessment  ratings 
(OAR)  have  been  included  in  order  to  allow  for  a  comparison 
between  this  unigue  conglomeration  of  data  and  individual 
selection  methods. 

Dependent  variables.  All  dependent  variables  reflected 
a  measure  of  achievement  at  an  administrative  level.  In  a 
review  of  selection  research,  Korman  (1968)  used  at  least 
fourteen  different  measures  of  achievement:      (a)  supervisor 
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ratings,  (b)  turnover,  (c)  objective  performance  data,  (d) 
overall  ratings,  (e)  promotion  rate,  (f)  administrative  level, 
(g)  tenure,  (h)  suggestion  plan  activity,  (i)  pay,  (j) 
mobility,  (k)  position  level  changes,  (1)  termination,  (m) 
critical  incidents,   and  (n)   combat  ratings. 

For  the  purpose  of  this  study  these  have  been  condensed 
to  the  following  seven  measures: 

1.  administrative  level  achieved, 

2.  salary  level  attained, 

3.  supervisor  ratings, 

4.  number  of  years  serving  in  an  administrative 
position  within  the  same  organization, 

5.  achievement  of  tenure, 

6.  objective  performance  data,  and 

7.  promotion  rate. 

Research  reviewed  that  defined  administrative 
achievement  as  measured  by  (a)  admissions  personnel  ratings, 
(b)  success  in  being  hired,  or  (c)  performance  success  in 
subseguent  training  programs  were  not  used  in  this  study. 
These  criterion  were  judged  to  be  too  far  removed  from  any 
measurement  of  actual  on-the-job  performance. 

Additional     variables.  Additional     variables  were 

classified  to  aid  in  the  meta-analyses  of  both  the  content  and 
the  methodology  of  the  research  studies  used  in  this 
dissertation.     In  order  to  understand  better  the  validities 
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involved  in  administrative  personnel  research  the  following 
aspects  of  each  study  were  recorded  and  coded: 

1.  the  year  the  study  was  published  (where  the  year 
was  not  given,  the  year  of  the  report  was  used) , 

2 .  the  type  organization  the  sample  was  derived  from 
(corporations,  educational  institutions,  military 
services,  or  public  administration) , 

3.  the  number  of  subjects  in  the  sample, 

4.  the  types  of  subjects  used,  their  job  levels  and 
sexes, 

5.  the  number  of  years  between  measurement  of  the 
independent  and  dependent  variables, 

6.  an  ordinal  measure  of  range  restriction, 

7.  the  type  validity,  predictive  or  concurrent, 

8.  the  presence  or  absence  of  criterion 
contamination,  and 

9.  whether  the  independent  variable  was  derived  from 
an  assessment  center. 

Procedures  for  the  Meta-Analysis 
The  methods  for  synthesizing  the  results  of 
correlational  studies  that  report  on  the  relationship  between 
two  variables  are  rather  straightforward.  Essentially,  the 
average  of  the  correlations  between  the  two  variables  that 
examine  the  same  research  guestion  across  separate  research 
studies  is  obtained  and  analyzed.  Reports  of  administrative 
selection  methods  usually  expressed  findings  in  the  form  of 
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a  correlation  coefficient.  Glass  (1977)  believed  that  studies 
which  were  combined  must  have  similar  aspects.  He  was  aware 
that  Light  and  Smith  (1971)  stated  that  the  independent  and 
dependent  variables  analyzed  had  to  be  measured  in  the  same 
way,  or  in  a  manner  which  could  be  converted  into  the  scales 
used  in  the  majority  of  studies.  Procedures  used  in  this 
study  followed  those  described  by  Glass  (1977)  for  the  meta- 
analysis of  a  sample  of  independent  correlational  studies. 
Calculation  of  Data 

Most  studies  contained  more  than  one  correlation 
coefficient.  Thus,  each  individual  correlation  within  a  study 
was  treated  as  a  separate  record  as  long  as  at  least  one  of 
the  variables  measured  was  different  from  all  others.  Each 
record  was  recorded  in  a  database  management  program  which 
contained  fields  for  each  of  the  variables  listed  above.  The 
answers  to  the  guestions  posed  in  the  Statement  of  the  Problem 
were  derived  from  taking  cross-sections  of  the  data  within  the 
variables  by  sorting  studies  based  upon  the  fields  listed 
above.  The  following  meta-analytic  procedures  were  then 
followed  to  better  understand  the  data. 

Conversion  to  a  common  metric.  The  correlations  in  each 
study  were  first  converted  into  a  common  metric  in  order  to 
determine  their  relative  effect  sizes.  The  effect  size  is 
simply  a  measure  of  the  strength  of  the  relationship  between 
two  variables.  In  this  dissertation,  the  Pearson  product- 
moment  correlation  was  used  as  the  common  measure  of  effect 
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size.  Research  studies  reviewed  here  which  were  not 
originally  reported  using  this  correlation  were  all  converted 
to  their  equivalent  Pearson  product -moment  correlations  using 
the  formulas  given  in  Wolf  (1986) . 

Averaging  correlations  across  studies  and  variables. 
From  the  correlation  coefficients  derived  by  each  query,  means 
and  standard  deviations  were  calculated  for  the  relationships 
between  selection  methods  and  administrative  achievement  using 
formulas  (1)  and  (2) .  According  to  Wolf  "the  average  of  the 
correlations  between  the  two  variables  that  examine  the  same 
research  question  across  separate  research  studies  is 
obtained"  (1986,  p.  28) .  Means  are  typically  calculated  by 
averaging  the  raw  Pearson  product-moment  correlation 
coefficients  (r)  using  the  formula: 

Er 

r  =    (1) 

n 

r  =  mean  correlation 
r  =  Pearson  correlation 

n  =  number  of  correlation  coefficients  combined 

(Wolf,   1986,   p. 29) 

The  standard  deviations  of  the  means  were  then 
calculated  using  the  formula: 


a  =  square  root  of   N  (2) 

N 

a  =  standard  deviation 


Ill 

N  =  number  of  cases 

r  =  correlation  coefficient 

However,  with  the  widely  varying  sample  sizes  found  in 
the  research  studies  reviewed,  from  20  to  8,885  subjects,  it 
was  deemed  necessary  to  weight  all  mean  correlations  by  sample 
size.     This  was  done  according  to  the  formula: 

r  =    (3) 

r-  =  correlation  coefficient  in  study  i 
Nj  =  number  of  persons  in  study  i 

(Hunter,   Schmidt,   &  Jackson,   1982,  p.  41) 

Elimination  of  sampling  error.  When  an  inference  is 
made  from  a  sample  to  a  population  a  certain  amount  of  error 
is  involved  because  random  samples  can  be  expected  to  vary 
from  one  to  another.  Sampling  error  is  defined  as  the 
difference  between  a  population  parameter  and  a  sample 
statistic.  Researchers  depend  upon  sample  statistics  to 
estimate  population  parameters,  thus,  the  knowledge  of  how 
samples  are  expected  to  vary  from  populations  is  a  basic 
element  in  inferential  statistics  (Ary,  Jacobs  &  Razavieh, 
1979 ,  p.   136) . 


112 

Sampling  error  causes  the  variance  across  studies  to 
increase  relative  to  the  variance  of  the  population 
correlations.  The  effect  of  sampling  error  on  the  variance 
is  to  add  a  known  constant,  which  is  called  the  sampling  error 
variance.  To  eliminate  the  effect  of  sampling  error  from  a 
meta-analysis  the  measure  of  sampling  error  variance  is 
subtracted  from  the  observed  variance.  This  is  done  using  the 
following  formulas  for  finding  the  variance  of  correlations 
across  samples: 


SCN^r,  -  r] 

Sz  =   

r  ENi 

S2  =  estimate  of  variance  of  r 

r 

Ni  =  number  of  persons  in  study  i 

ri  =  correlation  coefficient  in  study  i 

r    =  mean  correlation 


(1  -  r2)2K 

a2  =    (5) 


e 


EN 


a    =  sampling  error 

e 

r  =  mean  correlation 

K  =  number  of  studies 

N  =  number  of  persons  in  study 


Thus,  from  formulas  (4)  and  (5)  we  see  that  the  variance  of 
the  population  correlations  is  estimated  by: 
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est  a2  =  S2  - 


(1  -  r2)2K 


N 


(6) 


a  =  estimated  variance  in  population  correlations 
S2  =  estimate  of  variance  of  r 

r 

r  =  mean  correlation 

K  =  number  of  studies 

N  =  number  of  persons  in  study 


(Hunter,   Schmidt,    &  Jackson,    1982,   p.  44) 


Adjusting  for  error  of  measurement.  To  eliminate  the 
artifacts  due  to  error  of  measurement,  reliabilities  of  the 
variables  are  computed  according  to  reliability  data  given  in 
the  research.  The  formulas  for  adjusting  for  error  of 
measurement  are: 

a  =    (7) 

N 

a  =  mean  product  variable 

R  =  measure  of  reliability 

N  =  number  of  persons  in  study 

(Hunter,   Schmidt,   &  Jackson,   1982,   p.  82) 

Then  a  correlation  closer  to  the  true  score  can  be  calculated 
by: 


xy 

a  b 


(8) 
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PTU  =  mean  true  score 

a  =  mean  product  variable 

b  =  mean  product  variable 

(Hunter,   Schmidt,    &  Jackson,    1982,   p.  83) 

Reliability  data  were  not  found  reported  in  all  of  the 
research  studies  reviewed.  Reliability  data  were  presented 
in  several  studies  for  the  independent  variables  used  (Bray 
&  Grant,  19  6  6;  Dicken  &  Black,  1965;  Grant  &  Bray,  19  69; 
Grant,  Katkovsky,  &  Bray,  1967;  Hollander,  1965).  The 
reliability  coefficient  of  .82,  used  in  this  study,  was 
derived  by  averaging  the  reliability  findings  of  these  five 
research  studies. 

One  reference  was  found  for  reliability  data  on  the 
outcome  variable  of  supervisor  ratings.  This  coefficient  was 
.60  (King,  Hunter,  &  Schmidt,  1980).  This  coefficient  has 
been  used  for  the  reliability  coefficient  of  the  combination 
of  outcome  variables  in  this  study. 

Conclusion 

The  results  of  the  application  of  the  above  procedures 

to  the  data  derived  from  the  52  research  studies  reviewed  are 

presented  in  Chapter  IV  to  answer  the  questions  posed  in  the 

Statement  of  the  Problem.     As  Glass  has  stated: 

Most  of  us  were  trained  to  analyze  complex 
relationships  among  variables  in  the  primary 
analysis  of  research  data.  But  at  the  higher 
level,      where  variance,      nonunif ormity  and 

uncertainty    are    no    less    evident,    we    too  often 


substitute  literary  exposition  for  quantitative 
rigor.  The  proper  integration  of  research 
requires  the  same  statistical  methods  that  are 
applied  in  primary  data  analysis.     (1976,  p.  6) 


CHAPTER  IV 

PRESENTATION  OF  THE  META-ANALYSIS  OF  THE  DATA 

Introduction 

The  Korman  (1968)  study  combined  with  comments  by  Howard 
(1974)  provided  a  framework  for  this  study  of  selection 
methods  for  administrative  personnel,  their  predictive 
validities,  and  guidelines  for  administrative  personnel 
selection  in  public  community  colleges.  Hunter,  Schmitt,  and 
Jackson  (1982)  supplied  the  statistical  technique,  meta- 
analysis, by  which  one  could  quantify  the  answers  to  the 
questions . 

In  the  present  study  answers  to  the  following  questions 
were  sought : 

1.  What  methods  for  the  selection  of  administrative 
personnel  are  commonly  used  by  corporations,  public 
administration  agencies,  the  military  services,  and 
institutions  of  higher  education? 

2 .  What  methods  for  the  selection  of  administrative 
personnel,  derived  from  a  meta-analysis  of  selection  research, 
are  significantly  related  to  achievement  in  administrative 
positions? 
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3.  What  information,  derived  from  a  meta-analysis  of  the 
methodologies  of  selection  research,  are  useful  in 
interpreting  the  findings  of  this  study? 

4.  What  guidelines  can  be  derived  from  this  research  for 
improving  personnel  selection  methods  for  administrators  of 
public  community  colleges? 

In  the  study,  the  findings  of  research  of  administrative 
personnel  selection  methods  for  the  years  1962  through  1985 
inclusive  were  investigated.  This  study  built  upon  a 
classification  system  used  by  Korman  (1968)  and  was  carried 
further  by  the  use  of  meta-analysis.  Since  summary  statistics 
could  be  standardized  into  standard  effect  size  measures,  it 
was  possible  to  combine  the  guantitative  findings  of  different 
studies  and  to  identify  those  selection  methods  which  showed 
validity  in  predicting  administrative  achievement. 

Initially,  approximately  3  00  research  studies  on 
administrative  personnel  selection  methods  were  surveyed. 
Many  reports  contained  outcome  criterion  measures  which  were 
not  usable  considering  the  delimitations  of  this  study. 
Finally,  52  research  studies  which  contained  a  total  of  128 
correlation  coefficients  were  identified  as  meeting  the 
criterion  for  this  study. 

In  this  chapter,  the  results  of  the  meta-analysis  of  the 
data  derived  from  the  52  research  studies  reviewed  are 
presented.     Further  reporting  of  the  studies  is  based  on  the 
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128  correlation  coefficients  rather  than  discussing  the 
studies  individually. 

Selection  methods  are  abbreviated  in  the  tables  as 
follows:  (a)  aptitude  and  intelligence  measures  (APT) ,  (b) 
personal  interviews  (INTER) ,  (c)  job-related  skills  indicators 
(JOB) ,  (d)  psychological  attribute  indicators  (PSYCH) ,  (e) 
value  systems  assessments  (VAL) ,  (f)  biographical  information 
(BIO) ,  (g)  peer  ratings  (PEER) ,  (h)  self-appraisals  (SELF) , 
and  (i)  overall  assessment  ratings  (OAR) .  Outcome  criteria 
are  abbreviated  in  the  tables  as  follows:  (a)  supervisor 
ratings  (SUPRAT) ,  (b)  administrative  level  achieved  (ADLEV) , 
(c)  promotion  rate  (PRO) ,  (d)  salary  level  attained  (SAL) ,  (e) 
number  of  years  serving  in  an  administrative  position  within 
the  same  organization  (NUMYR) ,  and  (f )  a  combination  of  any 
of  the  above  criteria  (COMB) . 

A  General  View  of  the  Research 

In  Tables  1  and  2,  descriptive  analyses  of  the  research 
are  displayed.  The  freguency  of  reporting  of  correlations  by 
year  and  selection  method  is  indicated  in  Table  1.  The  nine 
independent  variable  classifications  are  those  used  throughout 
this  dissertation.  The  greatest  number  of  correlations  (16) 
was  reported  in  1984,  while  14  were  gathered  in  1977,  13  in 
1965,  and  12  in  1970.    Other  years  yielded  an  average  of  four 
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Table  1 


Frequency  of  Independent  Variable  Classifications  of 
Correlations  by  Reporting  Year 


Selection  method 


Year  APT     INTER     JOB     PSYCH     VAL     BIO     PEER     SELF  OAR 


1962 

1 

2 

1 

1963 

1 

1 

2 

1964 

2 

2 

2 

1 

1965 

5 

3 

4 

1 

1966 

1 

1 

1 

2 

1 

2 

1967 

1 

3 

1 

1 

1 

1968 

6 

1 

1969 

2 

1 

1 

1 

1970 

1 

2 

2 

1 

1 

1 

1 

3 

1971 

1972 

1 

1 

1 

2 

1973 

1974 

1 

1 

1 

1 

1975 

3 

1 

2 

2 

1 

2 

1976 

1 

1 

1 

1977 

2 

9 

2 

1 

1978 

1 

1 

1 

1 

1 

1979 

1980 

1981 

1 

1982 

1 

1983 

1 

1984 

2 

2 

2 

2 

2 

3 

3 

1985 
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studies  each  with  five  years  yielding  none.  Research  on 
administrative  personnel  selection  methods  may  have  been  done 
during  these  years,  however,  any  such  research  did  not  meet 
the  fairly  strict  delimitations  of  this  study. 

The  study  of  the  validity  of  psychological  attribute 
indicators,  such  as  motivation,  has  received  by  far  the  most 
attention  in  research.  Of  the  total  of  128  correlation 
coefficients  found,  37  were  derived  from  research  of 
psychological  attributes.  Clearly  there  was  an  interest  in 
finding  the  motivations  and  attitudes  held  by  successful 
administrators . 

The  second  most  freguently  researched  administrative 
selection  method  was  aptitude  and  intelligence  measures. 
Researchers  have  persisted  in  studying  the  validity  of 
intelligence  measures  despite  the  fact  that  administrative 
position  applicants  were  already  highly  preselected  on  this 
trait.  Possible  reasons  for  this  emphasis  in  the  research 
are  simply  the  availability  of  the  data  and  the  relative 
reliability  of  intelligence  measures. 

The  selection  methods  of  job-related  skills  indicators 
and  assessment  center  overall  assessment  ratings  are  almost 
egually  represented  in  the  research.  This  eguality  in  study 
is  most  likely  due  to  the  fact  that  most  assessment  centers 
rely  heavily  on  job-related   skills   indicators,    so   for  each 
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overall  assessment  rating  at  least  one  measure  of  job-related 
skills  was  also  recorded. 

The  remaining  administrative  personnel  selection  methods 
were  represented  relatively  egually  in  the  research  with 
between  5  and  11  correlation  coefficients  each  recorded  in 
this  study.  Interestingly,  these  "least  studied"  methods 
contain  all  three  of  the  methods  which  are  used  most 
freguently  in  actual  administrative  personnel  selection. 

A  review  of  the  data  derived  from  the  research  studies 
revealed  that  91%  of  the  administrative  personnel  selection 
research  was  done  within  corporations,  none  within  public 
administration  agencies,  6%  within  the  military  services,  and 
3%  within  educational  institutions. 

In  Table  2,  descriptive  data  are  presented  for  the 
distributions  of  the  validity  coefficients  before  weighting 
for  sample  sizes  or  correcting  for  sampling  error  and 
reliability.  The  mean  correlation,  median,  standard 
deviation,  and  range  are  shown  for  each  selection  method.  The 
mean  and  standard  deviation  were  calculated  using  formulas  (1) 
and  (2)   respectively  from  Chapter  III. 

It  is  clear  that  the  correlations  which  make  up  each  mean 
correlation  have  a  wide  range.  Typically  the  standard 
deviations  from  the  mean  egual  approximately  50%  of  the  mean. 
For  most  selection  methods,  the  distributions  were  fairly 
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Table  2 

Mean  Correlations,  Medians,  Standard  Deviations,  and  Ranges 
of  Uncorrected  Data 


Selection 
method 


Mean  correlations,  medians,  standard 
deviations,  and  ranges 


OAR 

JOB 

VAL 

INTER 

APT 

PEER 

BIO 

PSYCH 

SELF 


—  X     +  —  )  — 

[  (  X+  )  ] 

[  (—  x  +  )--] 

[  (—  X+  — )-] 

[  — (—  +x  )  ] 

[  (  x  +  — )  ] 

 (___  +     x  )  ] 

[-(---  +x  )  ] 

[-(---  x  +]) 


-.10      .00      .10      .20      .30      .40      .50      .60  .70 


Note.      [   ]  =  range,     X  =  mean,     +  =  median,      (   )   =  SD. 


symmetric.  For  self-appraisals,  however,  the  distribution  is 
severely  skewed  to  the  left. 

Administrative  Personnel  Selection  Methods 
In    this    section,     administrative    personnel  selection 
methods  are  discussed  in  terms  of  which  methods  have  been  most 
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commonly  used  in  personnel  selection  overall  and  in  each 
specific  areas  of  (a)  corporations,  (b)  public  administration, 
(c)  military  services,  and  (d)  education.  The  frequency  of 
use  of  each  selection  method  was  compared  to  its  ability  to 
predict  administrative  success.  The  statistical  data  in  this 
section  were  derived  by  calculating  mean  correlations  across 
selection  methods  and  by  then  weighting  these  correlations  by 
sample  size  using  formula  (3)  presented  in  Chapter  III.  The 
complete  meta-analysis  of  each  method,  in  which  corrections 
were  made  for  sampling  error  and  reliability,  is  presented  in 
a  later  section. 

Most  Commonly  Used  Administrative  Personnel  Selection  Methods 
The  most  commonly  used  selection  methods  for 
administrative  personnel  remain  essentially  the  same 
throughout  all  types  of  organizations  considered. 
Biographical  information  was  used  to  weed  out  those  who  did 
not  meet  minimum  qualifications  for  the  position.  Self- 
appraisals  were  often  provided  for  administrative  level 
positions  in  the  form  of  cover  letters  to  resumes.  The 
military  services  and  public  administrative  agencies  were 
likely  to  follow  up  with  tests  of  aptitude  and  intelligence. 
Almost  all  organizations  included  a  personal  interview.  Peer 
ratings  were  often  provided  in  the  form  of  references  when 
required.        Measures    of    psychological    attributes,  values 
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systems,  job-related  skills,  and  overall  assessment  center 
ratings  were  less  often  used. 

One  of  the  most  commonly  used  administrative  personnel 
selection  methods  was  the  biographical  information  method. 
This  information  is  presented  in  the  form  of  application 
blanks  and  resumes  (Levine  &  Flory,  1975) .  Its  popularity 
stemmed  from  the  accepted  notion  that  the  best  predictor  of 
future  behavior  is  past  behavior  (Childs  &  Klimoski,  1986) . 
In  most  cases,  biographical  information  has  been  used  to  help 
determine  if  an  individual  meets  the  minimum  qualifications 
for  a  particular  job.  Most  often  this  has  been  the  way  in 
which  a  large  group  of  job  applicants  was  reduced  to  a 
manageable  number  for  further  consideration. 

Another  often  used  method  has  been  the  personal 
interview.  According  to  Carlson  (in  Levine  &  Flory,  1975) 
approximately  50%  of  those  persons  filing  applications  or 
resumes  are  ultimately  interviewed.  Spriegel  and  James  (in 
Ulrich  &  Trumbo,  1965)  revealed  that  99%  of  852  firms  surveyed 
had  interviewed  applicants  before  hiring.  This  method  may  be 
combined  with  tests  of  aptitude  or  job-related  skills,  but  at 
the  administrative  level  often  it  is  not.  According  to  Jablin 
(1975)  , 

the  interview  should  serve  as  the  key  predictor  of 
the  prospective  employee's  ability  to  fit  into  the 
organizational  environment.  In  other  words,  the 
interview  would  not  evaluate  the  candidate's 
occupational  skills  and  aptitudes  but  rather  his/her 
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potential     to     fit     into     the  social-functional 
environment  of  the  job  and  organization.      (p.  5) 

Until  recently,    evidence  for  the  validity  of  the  interview 

was  particularly  low   (Arvey,  Miller,  Gould,   &  Burch,   1986) . 

Peer  ratings  have  been  frequently  collected  in  the  form 
of  references  and  recommendations.  Research  on  peer  ratings 
may  be  generalized  to  include  references  given  by  an 
applicant's  peers  or  co-workers.  According  to  Korman,  (1968) 
peer  ratings  are  "impressions  gathered  from  interactions  of 
an  equal,  non-supervisor-subordinate  nature"  (p.  313) .  No 
research  was  found  which  addressed  references  per  se. 

The  remaining  selection  methods  were  used  less  frequently 
than  the  ones  mentioned  above.  At  progressively  higher  levels 
of  management,  fewer  inquiries  have  been  made  into  an 
applicant's  values,  motivations,  and  intelligence  other  than 
what  can  be  gathered  in  the  most  commonly  used  methods.  Job- 
skills  indicators  and  assessment  center  processes  have  been 
used  only  in  organizations  which  process  a  fairly  large  amount 
of  applicants. 

The  selection  methods,  and  the  order  in  which  they  are 
employed,  vary  slightly  from  one  type  organization  to  another. 
Research  done  in  corporations  pervades  the  literature.  Most 
of  this  research  has  been  conducted  by  university-based 
researchers  who  either  have  been  hired  to  do  research  for  the 
company  or  have  been  doing  personnel  selection  research  for 
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a  doctoral  dissertation.  Little  research  has  been  done  with 
public  administrative  agencies.  None  of  the  research  which 
fits  the  delimitations  of  this  study  was  conducted  in  public 
administration . 

A  moderate  amount  of  personnel  selection  research  had 
been  completed  within  the  military  services;  however,  most  of 
this  research  did  not  meet  the  parameters  of  this  study.  Very 
little  research  has  been  done  within  educational  institutions. 
Although  most  administrative  personnel  selection  research  has 
been  conducted  by  university-based  researchers,  the  subjects 
of  their  studies  were  almost  always  drawn  from  employees  of 
a  corporation  and  not  from  educational  institutions.  This 
research  was  often  funded  by  the  individual  corporation. 
Validities  of  Selection  Methods  and  Their  Frequency  of  Use 

All  personnel  selection  methods,  combined,  correlated 
significantly  at  the  .05  level  with  administrative  achievement 
with  a  mean  correlation  coefficient  of  r  *  .22  (a  =  .14), 
calculated  according  to  formulas  (1)  and  (2)  in  Chapter  III, 
and  r  =  .25  when  weighted  for  sample  size  according  to  formula 
(3) .  Thus,  with  combined  samples  the  correlations  were 
significant  meaning  that  personnel  selection  methods,  taken 
as  a  whole,  were  predictive  of  administrative  achievement. 
For  sample  sizes  of  at  least  100  subjects,  the  .05 
significance  level  is  set  at  r  =  .195  (Ary,  Jacobs,  & 
Razavieh,     1979) .         However,     the     frequency     of     use  of 
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administrative  selection  methods  appeared  to  bear  little 
relationship  to  each  method's  ability  to  predict  achievement. 
The  correlation  coefficients  used  here  were  mean  correlations 
taken  across  each  selection  method  and  weighted  for  sample 
size  (see  Table  3) . 

The  assessment  center  overall  assessment  rating  was 
ranked  highest  in  validity  of  the  selection  procedures.  A 
total  of  14,224  subjects  were  included  in  14  studies  which 
yielded  16  correlation  coefficients  involving  assessment 
center  overall  assessment  ratings.  The  mean  correlation  of  r 
=  .32  (a  =  .14)  was  raised  to  r  =  .37  when  weighted  according 
to  sample  size.  Although  this  method  is  highly  valid  it  has 
been  rarely  used  outside  large  corporations  because  of  the 
costs  in  time  and  money. 

Peer  ratings  correlated  with  administrative  achievement 
showing  a  mean  coefficient  of  r  =  .28  (a  =  .14).  When 
weighted  for  sample  size,  this  correlation  dropped  to  r  =  .20. 
This  selection  method  dropped  from  its  second  place  ranking, 
according  to  strength,  to  sixth  place  when  weighted. 
Essentially  related  to  references  and  recommendations,  the 
predictive  ability  of  this  method  indicated  that  references 
and  recommendations  may  have  some  validity  in  the  selection 
process.     Unfortunately,  considering  the  legal  problems  that 
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Table  3 

Rankings  of  Validities  of  Selection  Methods  With  and  Without 
Weighting  for  Sample  Size 

Rank  Method  Unweighted  r        Method  Weighted  f 


1 

OAR 

.32 

OAR 

.  37 

2 

PEER 

.28 

JOB 

.30 

3 

JOB 

.25 

VAL 

.25 

4 

BIO 

.23 

INTER 

.24 

5 

INTER 

.20 

APT 

.21 

6 

VAL 

.20 

PEER 

.20 

7 

PSYCH 

.  19 

BIO 

.18 

8 

APT 

.  19 

PSYCH 

.15 

9 

SELF 

.  17 

SELF 

.  10 

truthfulness  in  references  might  bring,  there  may  be  a  large 
gap  between  the  validity  of  this  method  when  used  in  research 
as  opposed  to  its  use  in  actual  hiring. 

Job-related  skills  indicators  were  ranked  third  in 
strength  of  validity.  The  correlation  coefficient  of  this 
method  was  strengthened  from  r  =  .25  (o  =  .13)  to  r  =  .30  by 
weighting  for  sample  size.  Thus,  it  was  ultimately  ranked  as 
the  second  most  valid  method  after  weighting.    This  method  has 
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been  rarely  used  outside  an  assessment  center  program.  The 
measurement  of  job-related  skills  indicators  usually  involves 
rating  a  performance  on  tests  of  job  skills,  and  this  involves 
costs  in  both  time  and  money  in  setting  up  reliable 
performance  tests  of  administrative  skills. 

Biographical  information  correlation  coefficients  showed 
a  decrease  in  predictive  strength  from  r  =  .23  (a  =  .16)  to  r 
=  .18  after  weighting  for  sample  size.  Thus,  it  slipped  from 
fourth  place  to  seventh  place  in  predictive  validity.  This 
is  a  mediocre  ranking  for  such  a  frequently  used  selection 
method. 

The  selection  method  of  personal  interviews,  was 
strengthened  when  weighted  for  sample  size.  Originally  ranked 
fifth  with  a  correlation  of  r  =  .20  (a  =  .10),  it  was  raised 
to  r  =  .24,  fourth  place,  when  weighted.  Thus,  at  least  one 
of  the  three  most  frequently  used  selection  methods  ranked  in 
the  top  half  in  predictive  validity. 

The  selection  method  of  values  systems  assessments  was 
strengthened  when  weighted  for  sample  size.  The  mean 
correlation  coefficient  was  raised  from  r  =  .20  (a  =  .09)  to 
r  =  .25.  Thus,  this  method  rose  from  sixth  to  third  in 
ranking  by  predictive  strength.  According  to  England  and  Lee 
(1974) ,  "it  is  evident  that  value  patterns  are  predictive  of 
managerial  success  and  could  be  used  in  selection  and 
placement  decisions.      A  number  of  legal  and  ethical  issues 
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must  be  considered,  however,  before  such  use  is  justified"  (p. 
418)  . 

Ranked  seventh  was  the  method  of  psychological  attribute 
indicators  with  a  mean  correlation  of  r  =  .19  (a  =  .11)  .  This 
correlation  was  lowered  to  r  =  .15  when  weighted  for  sample 
size.  Thus,  this  method  dropped  to  eighth  place  out  of  the 
nine  methods.  Measures  of  interests  have  not  been  used 
extensively  in  managerial  selection  (Nash,  1965) .  Often  it 
was  not  used  past  the  first-level  of  supervisory  selection. 

The  intelligence  and  aptitude  measures  selection  method 
originally  displayed  a  mean  correlation  coefficient  of  r  =  .19 
(a  =  .11)  which  was  strengthened  to  r  »  .21  by  weighting  for 
sample  size.  Its  ranking  was  raised  from  eighth  to  fifth 
place  by  this  increase  in  strength.  The  use  of  this  method 
has  become  increasingly  infreguent  as  the  higher  levels  of 
management  are  reached. 

Ninth  in  original  and  final  rankings  was  the  method  of 
self-appraisals  with  a  mean  correlation  coefficient  of  r  =  .17 
(a  =  .10).  Predictive  validity  was  dropped  to  r  =  .10  by 
weighting  for  sample  size.  Self-appraisal  remained  the  lowest 
ranked  administrative  personnel  selection  method.  This  method 
has  been  freguently  seen  in  the  form  of  some  resumes  and  in 
the  cover  letters  to  resumes. 

In  Table  4  the  frequency  of  use  of  each  selection  method 
compared  to  the  relative  strength  of  the  mean  effect  size  of 


each  method  is  presented.  The  mean  correlation  coefficient 
is  referred  to  as  an  estimate  of  "effect  size"  (Hunter  et  al., 
1982,  p.  145) .  Effect  size  ranges  were  based  upon  definitions 
for  small,  medium,  and  large  effect  sizes  as  stated  for  point 
biserial  correlations  in  Cohen  (1977) .  Thus,  as  shown  in 
Table  4,  the  two  selection  methods  which  showed  medium  effect 
sizes,  overall  assessment  ratings  and  job-related  skills 
indicators,  were  also  the  least  used. 

Table  4 

Selection  Method  Frequency  of  Use  Compared  With  Mean  Effect 
Size 


Frequency 

Mean  effect  size 

Low 

Medium 

High 

Frequent 

PEER 

INTER 
BIO 

Moderate 

SELF 

APT 
PSYCH 

Seldom 

VAL 
OAR 
JOB 

Two  of  the  three  most  frequently  used  methods, 
biographical  information  and  peer  ratings,  showed  low  effect 
sizes.    Only  one  frequently  used  method,  personal  interviews, 
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showed  a  medium  effect  size.  However,  the  validities  of  these 
selection  methods  have  not  been  corrected  for  sampling  error 
or  reliability,  and  these  corrections,  done  in  the  meta- 
analysis, were  found  to  increase  the  effect  sizes  of  the  mean 
correlations . 

Selection  Methods  Significantly  Related  to  Administrative 
Achievement:     A  Meta-analysis 
The  results  presented  in  this  section  were  derived  by  a 
meta-analysis  of  the  128  correlation  coefficients  sorted  by 
independent  variable,  selection  method.     Formulas  (4)  through 
(8) ,  presented  in  Chapter  III,  were  used. 

Eight  of  the  nine  personnel  selection  methods  were 
significantly  related  to  achievement  in  administrative 
positions.  The  one  method  which  was  not  significantly  related 
was  self-appraisals.  The  rankings  of  the  validities  of  the 
selection  methods  remained  the  same  after  these  corrections 
for  sampling  error  and  reliability  were  completed  in  the  meta- 
analysis (see  Table  5) .  The  fact  that  the  relative  rankings 
of  the  selection  methods  remained  the  same  after  these 
corrections  were  made  is  understandable.  Correction  for 
sampling  error  had  little  effect  due  to  the  facts  that  (a)  the 
amount  of  sampling  error  is  inversely  proportional  to  the 
sample  sizes,  and  (b)  the  sample  sizes,  when  combined  across 
studies,  were  large.  Adjustments  for  reliability  of 
independent  and  dependent  variables  effected  the  strengths  of 
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Table  5 

Selection  Method  Frequency  of  Use  Compared  with  Mean  Effect 
Size  Derived  Through  Meta-analysis 


Frequency 

Mean  effect  size 

Low 

Medium 

Hiqh 

Frequent 

INTER 

BIO 

PEER 

Moderate 

SELF 

PSYCH 
APT 

Seldom 

VAL 

OAR 
JOB 

the  correlation  coefficients  but  not  the  rankinq.  Since 
reliability  data  were  scantily  reported  in  the  research,  the 
same  reliability  data  were  used  for  each  meta-analysis  done 
on  each  independent  variable. 

In  the  followinq  discussion  of  the  results  of  the  meta- 
analysis, the  selection  methods  were  ranked  in  the  order  of 
their  predictive  validity,  from  hiqhest  to  lowest.  These 
data,  alonq  with  the  corrected  means,  are  presented  in 
Table  6.  Corrected  means  were  derived  throuqh  calculations 
done  usinq  formulas  (4)  throuqh  (8)  presented  in  Chapter  III. 
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Table  6 

Mean  Correlations,  Sample  Size  Weighted  Correlations,  and 
Corrected  Correlations  of  Selection  Methods 


Method  Mean  r  Weighted  r  Corrected  r 


OAR 

.32 

.37 

.52 

JOB 

.25 

.30 

.43 

VAL 

.20 

.25 

.  35 

INTER 

.20 

.24 

.34 

APT 

.  19 

.21 

.30 

PEER 

.28 

.30 

.28 

BIO 

.23 

.  18 

.26 

PSYCH 

.  19 

.  15 

.21 

SELF 

.  17 

.  10 

.  14 

Overall  assessment  ratings.  The  assessment  center 
overall  assessment  rating  was  by  far  the  "most  valid"  of  the 
selection  procedures.  A  total  of  14,224  subjects  were 
included  in  14  studies  which  yielded  16  correlation 
coefficients  involving  assessment  center  overall  assessment 
ratings.  The  mean  correlation  of  r  =  .32  (a  =  .14)  was  raised 
to   r  =  .37   when  weighted   according  to   sample   size.  These 
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figures  were  obtained  using  formulas  (1)  through  (3)  in 
Chapter  III.  The  correction  for  unreliability  caused  the 
weighted  mean  correlation  to  be  raised  from  r  =  .37  to  r 
=  .52,  which  was  calculated  according  to  formula  (8)  in 
Chapter  III. 

Job-related  skills  indicators.  A  total  of  16,517 
subjects  were  included  in  10  studies  which  yielded  12 
correlation  coefficients  involving  job-related  skills 
indicators.  The  mean  correlation  of  r  =  .25  (a  =  .13)  was 
raised  to  r  =  .30  when  weighted  according  to  sample  size. 
Correction  for  unreliability  caused  the  weighted  mean 
correlation  to  be  raised  from  r  =  .30  to  r  =  .43. 

Value  systems  assessments.  A  total  of  2,483  subjects 
were  included  in  the  4  studies  which  yielded  5  correlation 
coefficients  involving  value  systems  assessments.  The  mean 
correlation  was  r  =  .25  (a  =  .09)  when  weighted  for  sample 
size.  Correction     for     unreliability     caused     the  mean 

correlation  to  be  raised  from  r  =  .25  to  .35. 

Personal  interviews.  A  total  of  11,526  subjects  were 
included  in  6  studies  which  yielded  7  correlation  coefficients 
involving  personal  interviews.  The  original  mean  correlation, 
.20  (a  =  .10),  weighted  for  sample  size,  was  r  =  .24.  The 
correction  for  the  error  of  measurement  caused  the  mean 
correlation  to  be  raised  from  r  =  .24  to  .34. 
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Aptitude  and  intelligence  measures.  A  total  of  17,482 
subjects  were  included  in  15  studies  which  yielded  21 
correlation  coefficients  involving  aptitude  and  intelligence 
measures.  The  mean  correlation  of  r  =  .19  (a  =  .11)  was 
raised  to  r  =  .21  when  weighted  according  to  sample  size. 
Correction  for  unreliability  caused  the  weighted  mean 
correlation  to  be  raised  from  r  =  .21  to  r  =  .30. 

Peer  ratings.  A  total  of  3,413  subjects  were  included 
in  8  studies  which  yielded  10  correlation  coefficients 
involving  peer  ratings.  The  original  mean  correlation  of  r 
=  .28  (a  =  .14),  weighted  for  sample  size,  was  r  =  .20.  The 
adjustments  for  error  of  measurement  caused  the  mean 
correlation  to  be  raised  from  r  =  .20  to  .28. 

Biographical  information.  A  total  of  3,326  subjects  were 
included  in  8  studies  which  yielded  11  correlation 
coefficients  involving  biographical  information.  The  mean 
correlation  of  r  =  .23  (a  =  .16)  was  depressed  to  r  =  .18  when 
weighted  for  sample  size.  Correction  for  unreliability  caused 
the  mean  correlation  to  be  raised  from  r  =  .18  to  r  =  .26. 

Psychological  attribute  indicators.  A  total  of  7,503 
subjects  were  included  in  22  studies  which  yielded  37 
correlation  coefficients  involving  psychological  attribute 
indicators.  The  mean  correlation  of  r  =  .19  (a  =  .11)  was 
depressed     to     r     =  .15     when    weighted     for     sample  size. 
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Correction  for  unreliability  caused  the  weighted  mean 
correlation  to  be  raised  from  r  =  .15  to  r  =  .21. 

Self -appraisals.  A  total  of  2,847  subjects  were  included 
in  8  studies  which  yielded  9  correlation  coefficients 
involving  self-appraisals.  The  mean  correlation  of  r  =  .17 
(a  =  .10)  was  depressed  to  r  =  .10  when  weighted  for  sample 
size.  Correction  for  unreliability  caused  the  mean 
correlation  to  be  raised  from  r  =  .10  to  r  =  .14. 

The  corrected  correlations  of  the  nine  selection  methods 
showed  a  mean  correlation  of  r  =  .31  and  a  median  of  .30. 
The  validity  of  each  selection  method,  with  the  exception  of 
self-appraisals,  was  significant  at  the  .05  level.  The 
corrected  mean  correlations  presented  here,  using  formulas 
(4)  through  (8)  in  Chapter  III,  are  closer  to  the  true  mean 
correlations  than  are  the  unweighted  or  sample  size  weighted 
means.  However,  because  of  the  varied  methods  of  measuring 
both  the  independent  and  dependent  variables,  the  corrections 
done  here  could  only  partially  rectify  the  mean  correlation 
coefficients  in  the  direction  of  their  true  scores.  This  is 
evidenced  by  the  large  amount  of  variance  that  remained 
unaccounted  for  after  correction. 

Relationships  Between  Selection  Methods  and  Separate 
Measures  of  Administrative  Achievement 

A      review      of      the       literature      revealed  general 
dissatisfaction      with      all      measures      of  administrative 


138 

achievement  (Huck  &  Bray,  1976;  Korman,  1968;  Steel  &  Ovalle, 
1984)  .  What  is  called  for  is  an  outcome  measure  that 
accurately  and  objectively  assesses  an  administrator's  level 
of  achievement  on  the  job,  rather  than  someone's  subjective 
opinion  of  such  achievement.  No  plausible  solutions  to  this 
dilemma  were  suggested  in  the  literature.  In  fact, 
discrepancies  in  inter-dependent  variable  validities  were 
found  in  the  literature.  The  relationships  between  selection 
methods,  both  inclusively  and  separately,  with  the  six 
different  measures  of  dependent  variables  used  in  this  study 
are  shown  in  Table  7 . 

The  outcome  criterion  which  showed  the  highest  mean 
correlation  was  derived  from  an  average  of  only  two  widely 
disparate  correlation  coefficients  and,  thus,  may  not  have 
been  a  reliable  measure  of  achievement.  However,  the  measure 
of  administrative  achievement  which  was  a  combination  of  two 
or  more  of  the  other  outcome  criteria  had  a  relatively  high 
correlation,  r  =  .26,  and  was  based  upon  a  larger  sample  of 
correlations.  This  finding  substantiates  the  Campbell  et.  al 
(1970)  view  that  such  an  "overall  success  index"  (p.  166)  , 
while  not  explicitly  identifying  or  quantifying  actual 
managerial  behavioral  dimensions,  did  identify  managers 
differing  substantially  in  their  ability  to  utilize 
organizational  resources  effectively. 


Table  7 

Selection  Method  Correlation  to  Outcome  Criteria 


Method 

SUPRAT 

ADLEV 

PRO 

SAL 

NUMYR 

COMB 

OAR 

.  33 

.  38 

— 

.24 

— 

.15 

JOB 

.  19 

.  28 

.  11 

.26 

.49 

VAL 

.  16 

.  22 

.  32 

INTER 

.  13 

.27 

.  17 

.35 

.20 

APT 

.  13 

.21 

.  14 

.25 

PEER 

.32 

.29 

.  11 

.  33 

BIO 

.  14 

.  11 

.  17 

.  17 

.38 

PSYCH 

.24 

.  17 

.20 

.  13 

.19 

.08 

SELF 

.  16 

.26 

.01 

.24 

Mean  r  .22  .24  .19         .18  .27  .26 


Prediction  to  Successively  Higher  Levels  of  Administration 
Several  attempts  have  been  made  (Korman,  1968;  Kraut, 
1969;  Roadman,  1964)  to  determine  those  selection  methods 
which  can  predict  administrative  achievement  at  succeedingly 
higher  levels  of  management.  Roadman  (1964)  reported  that 
psychological  tests  had  been  found  useful  in  predicting  which 
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employees  would  be  successful  junior  or  middle  managers.  In 
such  tests,  the  mean  correlation  with  mental  ability  was 
definitely  higher  for  top  level  executives  than  for  middle 
managers,  but  the  large  amount  of  overlap  between  the 
distribution  of  scores  reduced  any  ability  to  make  predictions 
from  the  tests. 

Korman  (1968)  also  found  that  verbal  ability  tests  showed 
little  usefulness  for  predicting  managerial  performance  above 
the  first-line  supervisory  level,  because  the  typical 
managerial  applicant  population  was  already  highly  preselected 
on  abilities.  Biographical  information,  likewise,  showed 
"some  predictive  value"  for  first-line  supervisors  only 
(Korman,   1968,  p.  308). 

There  were  not  sufficient  correlation  coefficients 
present  in  this  study  to  do  a  complete  meta-analysis  in  which 
each  selection  method  was  averaged  by  each  level  of  management 
achieved.  However,  by  combining  all  selection  methods,  and 
comparing  them  to  the  different  administrative  levels 
achieved,  the  following  conclusions  were  found. 

Selection  methods  correlated  r  =  .36  with  attainment  of 
the  first-level  of  supervisory  achievement.  These  same 
methods  correlated  r  =  .27  and  r  =  .28,  respectively,  with 
the  mid-  and  high-levels  of  administrative  achievement.  Thus, 
the  results  confirmed  the  literature  findings  that  selection 
methods    can    more    accurately    predict    achievement    of  the 
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first-level  of  administration,  but  have  less  validity  in  the 
prediction  of  those  applicants  who  will  succeed  at  higher 
administrative  levels. 

Validity  of  the  Assessment  Center  Approach 

Howard  (1974)  stated  that,  because  of  the  expense 
involved,  psychologists  tried  to  develop  assessment  centers 
in  more  than  a  haphazard  way.  She  surmised  that  projective 
tests,  interviews,  and  job-related  exercises  were  designed 
with  care  toward  reliability  and  validity  for  each  particular 
organization.  The  results  of  this  study,  however,  showed 
little  overall  difference  between  the  validities  of  each  and 
all  selection  methods  when  used  within  an  assessment  center 
versus  when  used  independent  of  any  assessment  center. 

Table  8  shows  that,  of  the  eight  selection  methods  (OAR 
was  not  included) ,  half  displayed  higher  correlations  within 
an  assessment  center  and  half  without.  The  mean  correlations 
of  each  group,  r  =  .20  for  the  methods  within  an  assessment 
center,  and  r  =  .21  for  the  methods  taken  independent  of  any 
assessment  center,  were  almost  identical. 

Thus,  it  does  not  seem  that  the  assessment  center 
approach  added  any  value  to  each  separate  selection  method. 
Still  the  assessment  center  overall  assessment  rating  was,  in 
11  out  of  14  assessment  center  studies,  significantly 
correlated  with  administrative  achievement.  Several  reasons 
have    been    suggested     for    this    phenomenon.        Turnage  and 
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Muchinsky  (1984)  recognized  that  the  "Pygmalion  effect"  may 
play  a  part.  The  self  esteem  of  a  person  selected  for 
assessment  may  have  been  raised  substantially  and  "they  then 
seek  to  perform  both  in  assessment  and  later  on  the  job  at  a 

Table  8 

Correlations  of  Selection  Methods  Measured  Within  and  Without 


Assessment 

Center  Processes 

Selection 

Assessment 

Non— assessment 

method 

center 

center 

JOB 

.26 

.25 

PSYCH 

.  14 

.  20 

INTER 

.  18 

.  35 

APT 

.20 

.  18 

VAL 

.24 

.  18 

BIO 

.32 

.  18 

PEER 

.  12 

.  35 

SELF 

.13 

.  19 

Mean  r 

.20 

.21 

level  high  enough  to  substantiate  the  lofty  esteem  in  which 
they  are  perceived  to  be  held"   (p.   602) . 
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The  validity  of  the  job-related  skills  indicators  used 
in  every  assessment  center  process  has  been  suggested  as  a 
cause  of  the  high  validities  of  overall  assessment  ratings. 
Howard  (1974)  stated  that  "since  the  situational  tests 
represented  a  unique  contribution  to  the  process  and  were 
relied  upon  heavily,  it  was  usually  assumed  that  these  were 
what  made  the  difference,  although  specific  data  often  were 
not  reported"  (p.  126) .  The  results  of  this  study  bear  this 
out.  Thus,  the  job-related  skills  indicators  contributed  a 
great  deal  to  the  overall  assessment  rating  validity. 

A  final  reason  for  the  high  validity  of  the  assessment 
center  process  concerns  the  statistical  combination  of  the 
data  derived  from  each  selection  method  within  the  assessment 
center  process.  A  stepwise  multiple  regression,  used  to  form 
a  composite  of  the  results  of  all  selection  methods  within  the 
assessment  center,  has  been  used  to  almost  double  the  validity 
between  the  selection  process  and  the  outcome  criterion.  For 
example,  Wollowick  and  McNamara  (1969)  found  an  overall 
assessment  rating  of  r  =  .37  in  the  IBM  assessment  center 
study.  Using  all  the  assessment  data  in  an  actuarial  fashion 
produced  a  multiple  correlation  of  R  =  .62,  accounting  for  38% 
of  the  variance  (Howard,  1974) .  Thus,  using  the  most  common 
selection  methods,  and  combining  them  statistically,  is  an 
efficient  way  of  increasing  validity. 
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Correlation  of  Types  of  Interactions  Within  Selection 
Methods  and  Measures  of  Administrative  Achievement 

The  predictive  strength  of  each  type  of  interaction, 
written,  verbal,  or  performance,  required  from  an  applicant 
was  found  to  be  dependent  upon  the  type  of  selection  method 
used.  Selection  tests  which  required  primarily  written 
responses  showed  the  lowest  amount  of  correlation,  r  =  .19, 
when  weighted  for  sample  size.  Verbal  interactions  showed  a 
weighted  mean  correlation  of  r  =  .24.  Interactions  involving 
a  performance  of  a  task  by  the  applicant  showed  the  highest 
validity,  r  =  .31.  The  predictive  strengths  of  these 
interactions  were  not  independent  from  the  nine  selection 
methods  considered. 

The    verbal    interaction    correlation    coefficients  were 

derived  solely  from  the  studies  involving  the  selection  method 

of     personal      interviews.  The     performance  interaction 

correlation  coefficients  were  derived  predominantly  from  the 

selection  method   of   job-related   skills    indicators.  Thus, 

these  mean  coefficients  were  almost  identical  to  the  selection 

methods  from  which  they  were  drawn. 

Analysis  of  the  Research  Methods  Used  in  Administrative 
Personnel  Selection  Research 

Information     derived     from     a     meta-analysis     of  the 

methodologies  used  in  personnel  selection  research  were  found 

to  be  useful   in  interpreting  the  findings  of  the  research. 

The  problems  of  criterion  contamination,  restriction  of  range, 
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and  concurrent  validity  studies  are  discussed.  The  results 
of  an  analysis  of  the  merits  of  predictive  versus  concurrent 
validity  are  shown.  Results  are  given  for  an  analysis  of  the 
relative  advantages  of  various  time  intervals  between  the 
measurement  of  the  independent  and  dependent  variables  in  the 
predictive  studies  used. 
Criterion  Contamination 

The  extent  to  which  criterion  contamination  effects 
hiring  and  promotion  of  administrative  personnel  has  been  a 
constant  concern  of  researchers.  Howard  (1974)  stated  that 
problems  of  criterion  contamination  have  confounded  predictive 
validity  studies  when  personnel  decisions  based  upon  selection 
ratings  were  used  in  determining  future  promotions.  However, 
she  thought  that  even  where  these  data  were  used  to  aid  in 
initial  hiring  or  promotion  decisions,  the  effect  on  later 
promotions  was  not  large. 

The  results  of  the  present  meta-analysis  showed  that 
criterion  contamination  had  no  effect  on  the  combined 
validities  of  the  independent  variables.  When  weighted  by 
sample  size,  predictive  validity  studies  containing  criterion 
contamination  showed  a  mean  correlation  of  r  =  .25. 
Predictive  validity  studies  for  which  the  researchers  reported 
that  no  criterion  contamination  was  allowed  showed  a  mean 
correlation  of  r  =  .26.  Thus,  it  appears  that  the  fears  of 
researchers,     that    subseguent    measures    of  administrative 
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achievement  may  be  biased  based  upon  an  employee's  past  rating 
on  a  selection  procedure,  may  be  unwarranted. 
Range  Restriction 

According  to  the  results  of  the  present  meta-analysis, 
range  restriction  had  little  effect  on  the  combined  validities 
of  the  independent  variables.  One  would  expect  that,  as 
amount  of  range  restriction  increased,  the  ability  of  the 
correlation  statistics  to  determine  differences  between 
variables  would  decrease.  That  is,  as  range  restriction 
increases  it  is  expected  that  the  strengths  of  the  correlation 
coefficients  will  decrease.  This  was  not  proven  to  be  the 
case  in  this  study.  In  Table  9,  the  effects  of  range 
restriction  on  outcome  measures  of  administrative  achievement 
are  presented.  All  correlation  coefficients  reported  were 
weighted  by  sample  size. 

Studies  in  which  there  was  no  appreciable  range 
restriction,  in  which  subjects  were  in  nonmanagement 
positions,  showed  a  mean  correlation  coefficient  of  r  =  .25. 
Where  range  restriction  was  considered  low,  where  the  original 
subjects  were  low-level  managers,  the  mean  correlation 
coefficient  was  r  =  27.  Where  the  subjects  were  mid-level 
managers  at  the  time  of  measurement  of  the  predictor  variable, 
the  mean  correlation  coefficient  was  r  =  .23.     Where  range 
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Table  9 

Effects  of  Range  Restriction  on  Outcome  Measures  of 
Administrative  Achievement 

Effect  size  measurement 

Range  Small  Medium  Large 

restriction 


None 

.25 

Low 

.  27 

Medium 

.23 

High 

.  24 

restriction  was  considered  to  be  high,  meaning  the  subjects 
were  either  high-level  administrators  or  the  study  was  one  of 
concurrent  validity,  the  mean  correlation  coefficient  was 
r  =  .24. 

Predictive  Versus  Concurrent  Validity 

According  to  Stahl  (1983) ,  concurrent  validity  studies 
might  be  expected  to  show  higher  correlation  coefficients  than 
predictive  validity  studies.  It  has  been  suggested  that,  in 
a  concurrent  study  on  administrative  achievement,  a  supervisor 
completing  an  inventory  such  as  a  psychological  attributes 
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inventory,  might  reveal  that  he  or  she  has  many  of  the  traits 
needed  to  be  a  "successful"  manager.  He  or  she  may  not  have 
had  these  traits  originally  in  his  or  her  career,  but  may  have 
developed  and  reinforced  them  over  time.  However,  the  results 
of  this  meta-analysis  showed  significance  of,  and  little 
difference  between,  the  mean  correlations  of  the  predictive 
validity  and  the  concurrent  validity  studies,  r  =  .25  and  r 
=  .22,  respectively,  when  weighted  for  sample  size. 
Years  Between  Variable  Measurement 

According  to  Turnage  and  Muchinsky  (1984) ,  if 
organizations  collect  their  criterion  data  immediately  after 
assessment,  the  candidates  may  not  have  had  ample  time  to 
demonstrate  their  performance  on  the  job.  But,  if  too  much 
time  has  elapsed  between  the  assessment  and  the  performance 
evaluations,  many  extraneous  factors  may  account  for  the 
candidate's  level  of  performance. 

Data  from  this  study  showed  that  this  may  be  the  case. 
The  optimal  time  between  measurement  of  variables  may  be  from 
3  through  10  years.  Eighty-three  correlation  coefficients 
were  derived  from  studies  in  which  the  dependent  variable  was 
measured  less  than  3  years  after  the  measurement  of  the 
independent  variable.  These  included  all  coefficients  derived 
from  concurrent  validity  studies.  The  sample  size  weighted 
mean  correlation  was  r  =  .21.  Predictive  versus  concurrent 
validity  studies  showed  no  appreciable  difference  in  validity. 
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When    intervening    time    between    independent    and  dependent 

variable  measurement  was  increased  to  3  through  10  years,  the 

mean   correlation  was   raised   to   r   =  .30   when  weighted  for 

sample  size.     By  definition,  these  34  coefficients  were  drawn 

only  from  predictive  studies.    However,  when  years  between  the 

measurement  of  variables  was  greater  than   10  years,    the  10 

correlation  coefficients   concerned  yielded  a  weighted  mean 

correlation  of  r  =  .19. 

The  weighted  mean  correlation  of  r  =  .30,  for  the  studies 

where  time  between  variables  was  from  3  through  10  years,  was 

of    interest    to    this    researcher.        Explanations    for  this 

relatively    high    validity    were    sought.       The    most  logical 

explanation  would  be  for  an  included  study  to  contain  a  large 

sample  size  and  high  validity  coefficients.     Studies  by  Moses 

(1972)    met  these  requirements.      However,    when  the  weighted 

mean  coefficients  were  recalculated  omitting  the  studies  by 

Moses,  the  corresponding  mean  correlation  coefficient  was  only 

reduced  to  r  =  .26.     Therefore,   it  is  suggested  that  further 

investigation  be  made  into  determining  the  optimal  length  of 

time  needed  between  measurement  of  variables  in  administrative 

personnel  selection  research. 

Conclusions  of  the  Meta-analysis  of  the  Content  and 
Methodologies  of  Personnel  Selection  Research 

The   following   is   a   discussion   of   the   findings   of  the 

content    of    administrative    personnel    selection    research  as 
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pertains  to  the  predictive  validity  of  each  method.  Although 
a  sufficient  number  of  studies  and  correlation  coefficients 
were  found  to  gain  an  accurate  assessment  of  the  predictive 
validities  of  the  nine  selection  methods,  insufficient 
research/reporting  was  found  on  details  within  each  study 
needed  to  develop  moderator  variables  for  in-depth  research. 
Sufficient  data  were  provided  to  complete  the  review  of 
several  methodological  concerns.  A  summary  of  these  results 
is  presented  here. 
Content 

The  overall  assessment  rating  displayed  a  decidedly 
higher  correlation  than  all  other  methods.  This  correlation 
was  derived  from  a  combination  of  both  clinical  judgments  by 
trained  assessors  or  psychologists,  and  statistical  synthesis 
of  these  judgments  (Korman,  1968) .  Thus,  this  method  has 
validity-enhancing  aspects  rarely  found  in  other  methods. 

Job-related  skills  indicators,  although  lower  in  mean 
correlation  than  the  overall  assessment  rating,  have  been 
attributed  with  contributing  a  great  deal  to  the  overall 
assessment  rating  in  terms  of  validity  (Howard,  1974) .  This 
study  supported  this  view. 

Values  systems  assessments  showed  a  relatively  high 
correlation  with  administrative  success.  The  results  of  this 
study  supported  the  findings  of  England  and  Lee  (1974)  that 
"values  patterns  were  significantly  predictive  of  managerial 
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success"  (p.  411)  .  This  was  the  "least  researched"  of  the 
nine  selection  methods  in  this  study.  In  the  light  of  the 
relatively  high  validity  shown,  this  method  may  warrant 
further  research.  Care  should  be  taken  to  define  "values" 
precisely  and  not  consider  them  an  extension  of  psychological 
attributes  indicators. 

Ghiselli  (1966)  found  the  interview  to  have  at  least 
moderately  substantial  validity.  This  study  supported  his 
finding.  However,  there  has  been  concern  over  just  what 
factors,  or  moderator  variables,  contribute  to  the  validity 
of  the  personal  interview.  It  has  been  suggested  that  both 
structure,  such  as  with  an  interview  schedule  (Ghiselli) ,  and 
multiple  assessors  contribute  to  this  validity  (Carleton, 
1970) .  A  paucity  of  data  concerning  these  specific  factors 
in  the  seven  studies  reviewed  precluded  any  further  analysis 
within  the  selection  method  of  personal  interviews.  Further 
research  is  needed,  with  better  reporting  of  the  details 
involved,  to  allow  for  a  meta-analysis  for  the  effects  of 
moderator  variables  within  this  method. 

According  to  Korman  (1968) ,  ability  tests  have  shown 
little  usefulness  in  predicting  managerial  performance  above 
the  first-line  supervisory  level.  In  this  study,  aptitude 
and  intelligence  measures  fell  on  the  median  and  near  the  mean 
of  the  nine  selection  methods. 


152 

Korman  (1968)  found  that  peer  ratings  could  be  highly 
predictive  of  later  promotion  rate.  The  present  study  showed 
that  peer  ratings  were  significantly  related  to  later 
administrative  achievement.  According  to  Korman  (1968) ,  "when 
one  sees  how  consistently  this  is  an  effective  predictor  of 
officer  success,  it  is  surprising  that  the  method  has  not  been 
studied  more  thoroughly  in  the  industrial  context"   (p.   313)  . 

Biographical  information  ranked  third  from  the  lowest, 
but  was  still  significant,  as  a  selection  method.  Hinrichs 
(1969)  concluded  that  using  biographical  information  was  as 
effective  as  using  an  assessment  center  process  in  identifying 
management  potential.  This  study  does  not  bear  this 
conclusion  out,  however,  in  light  of  the  possibility  of 
obtaining  higher  correlation,  further  research  for  moderator 
variables  within  this  method  may  be  warranted. 

The  selection  method  of  psychological  attribute 
indicators  was  significant  in  predicting  administrative 
success.  However,  it  was  the  lowest  ranked  significant 
method.  This  was  a  poor  showing  considering  that  29%  of  the 
research  effort  has  been  put  toward  understanding  this  method. 
Thus,  this  study  substantiated  Korman 1 s  (1968)  view,  20  years 
later,  that  "there  seems  little  reason  for  thinking  that  we 
have  learned  much  about  the  psychological  variables  indicative 
of  managerial  behavior  insofar  as  these  variables  are 
determinable  by  objective  personality  inventories"   (p.  302) . 
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Self-appraisals  were  not  found  to  be  significantly 
associated  with  administrative  achievement.  With  the 
exception  of  Heneman  (1974) ,  most  researchers  found  that 
"individuals  rate  themselves  higher  than  they  are  rated  by 
comparison  groups"  as  such  they  may  lead  to  "inflated 
statements  of  qualification"  (Thornton,  1980,  p.  265,  p.  269). 
This  study  substantiated  this  view. 
Methodologies 

The  findings  of  the  meta-analysis  of  the  research 
methodologies  used  in  personnel  selection  research  revealed 
that  the  predictive  validities  of  the  predictor  variables  were 
affected  very  little  by  the  research  methodologies.  The 
presence  of  criterion  contamination  and  range  restriction 
produced  minimal  variation  across  studies.  The  results  of 
concurrent  validity  studies  and  predictive  validity  studies 
similarly  showed  little  deviation  from  one  another. 

These  findings  prompt  this  researcher  to  suggest  that 
personnel  selection  researchers  might  review  the  need  for  the 
tightly  controlled,  long-term  studies  which  are  often  called 
for  at  the  conclusion  of  concurrent  validity  studies.  Is 
there  empirical  evidence  that  such  studies  result  in  "more 
valid"  conclusions  than  concurrent  studies,  or  is  this  an 
assumption  that  has  been  perpetuated? 

In  Chapter  V  the  processes  currently  used  for  the 
selection  of  administrative  personnel  will  be  discussed.  The 
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extent  to  which  those  methods  now  in  use  for  the  selection  of 
administrative  personnel  in  public  community  colleges 
correspond  with  those  methods  which  show  the  highest 
correlation  to  administrative  achievement  will  be  explored. 
Guidelines,  derived  from  the  present  study,  for  improving 
personnel  selection  methods  for  administrators  of  public 
community  colleges  will  be  suggested. 


CHAPTER  V 

GUIDELINES  FOR  SELECTION  OF  ADMINISTRATIVE  PERSONNEL 
IN  PUBLIC  COMMUNITY  COLLEGES 

Introduction 

The  purpose  of  this  study  was  to  develop  a  set  of 
guidelines  for  improving  the  selection  of  administrators  of 
public  community  colleges.  The  findings  of  research  on 
administrative  personnel  selection  methods  were  surveyed  and 
a  meta-analysis  was  conducted  on  the  data.  In  this  chapter 
the  results  of  this  meta-analysis  are  applied  to  the  selection 
of  administrative  personnel  in  public  community  colleges.  The 
results  are  discussed  and  guidelines  for  improving  personnel 
selection  methods  for  administrators  of  public  community 
colleges  are  presented. 

The  Need  for  Valid  Personnel  Selection  Methods 

The  history  of  the  two-year  college  movement  was 
punctuated  by  a  tremendous  growth  spurt  in  the  1960s  and 
1970s,  which  brought  about  an  increased  demand  for 
administrators  to  staff  the  new  and  growing  colleges.  During 
this  period,  the  two-year  colleges  did  not  always  have  time 
to  "grow  their  own"  administrators.  This  historical  fact 
probably  helped  to  contribute  to  a  relatively  open  labor 
market  for  two-year  college  administrators  (Twombly,  1987,  p. 
14)  . 
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During  the  1980s,  following  this  period  of  unprecedented 
growth,  community  college  administrators  faced  a  new  set  of 
challenges.  Enrollments  plateaued;  state  governments  replaced 
local  communities  as  the  colleges'  principal  source  of 
funding;  steady  state  enrollments  and  greater  fiscal 
accountability  meant  that,  for  the  first  time,  community 
colleges  had  to  look  at  educational  outcomes  (Bernstein, 
1988) . 

The  diversity  of  the  student  body  of  the  1980s  added  to 
the  complexity  the  community  college  administrator  faced 
(Wattenbarger ,  Haynes,  &  Smith,  1982).  Wolf  (1985)  stated 
that  a  community  college  is  an  organization  in  which  "goals 
are  unclear,  technology  is  ambiguous,  and  participation  is 
fluid  among  faculty  and  administrators"  (p.  55)  .  Thus,  he 
suggested  that  the  best  way  to  achieve  organizational  outcomes 
was  to  place  emphasis  on  hiring  intelligent  people  as 
administrators.  As  Etzioni  pointed  out,  the  more  selective 
the  organization,  the  less  control  it  needs,  and  the  fewer 
problems  its  administrators  face  (in  Webb,  1983) . 
Present  Administrative  Personnel  Selection  Procedures 

In  the  past,  the  process  of  hiring  college  administrators 
has  usually  been  conducted  in  one  of  the  following  ways:  (a) 
the  implementation  of  a  search  committee  (Kelly  &  Nelson, 
1978) ,  (b)  by  placing  a  blind  ad  in  the  Chronicle  of  Higher 
Education .  or  by  (c)  using  an  employment  agency  or  executive 
search    firm     (Mottram,     1983) .        The    committee    or  person 
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responsible  for  the  search  described  the  position  to  be  filled 
and  wrote  an  appropriate  position  description.  The  position 
opening  was  often  advertised  in  the  Chronicle  of  Higher 
Education.  Whatever  the  method  used,  the  rule  has  been  for 
each  applicant  to  complete  an  application  form,  supply 
transcripts,  and  send  three  letters  of  recommendation 
(Scigliano,  1979).  Thus,  the  "first  cut"  made  by  the  search 
committee  has  been  based  upon  the  selection  methods  of 
biographical  information  and  peer  ratings  in  the  form  of  the 
completed  application  blank  and  recommendations.  Those 
applicants  who  passed  this  stage  were  asked  to  supply  previous 
employer  experience  affidavits,  copies  of  all  licenses  or 
certificates,  and  transcripts  from  all  institutions  attended 
(Scigliano,   1979) . 

According  to  Huegli  and  Eich  (1979) ,  the  most  crucial 
decisions  reached  in  the  search  process  were  those  which 
rested  on  the  results  of  on-site  interviews  with  candidates. 
Tremendous  weight  has  traditionally  been  placed  on  the  results 
of  the  interview.  The  candidate  pool  has  usually  been  pared 
down  to  only  a  handful  of  applicants  and  great  pressure  was 
on  to  select  one  of  them  or  begin  the  whole  process  over 
again.  After  this  final  screening,  the  recommendation  was 
made  by  the  committee  to  the  chief  administrative  officer  and 
board  (Huegli  &  Eich,   1979) . 

Thus,  according  to  the  findings  of  the  present  study,  the 
search    process    mainly    involves    the    selection    methods  of 
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biographical  information,  peer  ratings,  and  personal 
interviews.  These  three  methods  showed  medium  effect  sizes 
after  meta-analysis,  meaning  that  they  can  be  valid  predictors 
of  administrative  achievement. 

Two  selection  methods,  overall  assessment  rating  and  job- 
related  skills  indicators,  were  ranked  high  in  effect  size 
ratings,  but  are  seldom  used  in  selection  for  administration 
in  higher  education.  Only  one  use  of  an  assessment  center 
process  for  higher  education  was  identified  by  this  researcher 
(Brubaker,  1983)  .  It  was  not  used  for  selection,  but  for 
career  development. 

Three  additional  selection  methods,  psychological 
attributes  indicators,  aptitude  and  intelligence  measures,  and 
value  systems  assessements,  also  showed  medium  effect  sizes, 
but  the  use  of  these  methods  in  a  public  institution  may  lead 
to  legal  problems  if  the  hiring  institution  cannot  prove  that 
such  tests  are  specifically  job  related. 

One  method,  self-appraisal,  showed  a  low  effect  size. 
This  was  the  only  one  of  the  nine  selection  methods  that  was 
proven  to  be  invalid  in  the  prediction  of  administrative 
achievement . 

Legal  Constraints  Upon  Personnel  Selection 

In  selecting  administrators  in  public  community  colleges, 
the  persons  responsible  for  the  search  must  comply  with  legal 
constraints  which  affect  their  ability  to  apply  conclusions 
drawn     directly     from    the     research.         Although  personal 


159 

interviews  probably  will  continue  to  be  the  most  important 
part  of  the  process  (Webb,  1983) ,  testing  has  often  been  the 
most  controversial  part.  From  Griggs  v.  Duke  Power  Co. .  1971 
(in  Wisner,  1975)  several  rules  for  testing  in  personnel 
selection  were  derived.  Tests  must  be  used  only  as 
supplements  to  other  selection  methods  and  all  tests  must  be 
validated  within  the  organization  in  order  to  be  considered 
job-related  (Webb,  1983). 

The  case  of  Edward  L.  Kirkland  et  al..  Plaintiffs  V.  New 
York  State  Department  of  Correctional  Services  et  al.. 
Defendants  (73  LIV.  1548,  1974)  strengthened  the  requirement 
for  job-relatedness  of  selection  tests.  The  court  mandated 
that  every  measure  available  be  used  to  assure  that  tests 
would  not  discriminate  against  any  group  of  persons.  Through 
its  institution,  in  1972,  of  the  Higher  Education  Guidelines, 
the  federal  government  called  for  an  extension  to  recruitment 
and  hiring  activities  beyond  the  generally  perceived  network 
approach.  Dingerson,  Rodman,  and  Burns  (1985)  sought  to 
assess  what  changes  have  occurred  in  hiring  procedures  since 
implementation  of  the  Higher  Education  Guidelines.  They  found 
that  there  has  been  no  substantial  change  in  the  hiring  of 
candidates  from  underrepresented  populations  for  nationally 
advertised  positions.  There  is  simply  no  evidence  that  change 
is  underway   (Dingerson,   Rodman,   &  Burns,   1985) . 

Sunshine  laws  are  another  source  of  constraint  in  hiring 
in  public   community   colleges.      Some   search   committees  are 
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required  by  state  law  to  conduct  all  search  business  in 
public.  The  names,  resumes,  and  letters  of  recommendation  of 
all  candidates  must  be  available  to  the  press  and  general 
public  and  all  committee  votes  are  a  matter  of  public  record 
(McLaughlin,  1985) .  Thus,  the  problem  of  the  reliability  of 
the  data  provided  within  some  selection  methods  may  be 
suspect,  even  if  that  method  itself  has  been  proven  a  valid 
predictor  of  administrative  achievement. 
Guidelines  for  Selecting  Community  College  Administrators 

In  deriving  guidelines  from  the  research,  it  must  first 
be  determined  whether  such  guidelines  can  be  generalized  to 
aid  in  the  selection  of  community  college  administrators. 
Hammons  and  Ivery  (1987)  found  statistically  significant 
differences  between  the  tasks  of  five  presidents  of  community 
colleges  and  five  chief  executive  officers  of  corporations. 

Bare  (1986)  identified  two  unlike  "cores"  in  college  and 
university  administration.  He  found  that  work  in  the 
administrative  core  was  organized  in  a  mechanistic  manner  with 
an  emphasis  on  systematic  approaches  to  problem  solving. 

Bare  found  that  a  more  organic  form  of  work  organization 
prevailed  within  the  technical  core.  Departmental 
chairpersons  employed  the  more  consultative  leadership  style 
appropriate  to  academic  decision-making.  He  stated  that 
researchers  must  be  aware  that  the  core,  rather  than  the 
institution,  may  be  the  most  fruitful  level  for  theoretical 
analysis.       Thus,    it    may    be    the    case    that    the  personnel 
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selection  guidelines  from  the  research  can  be  applied  with 
differing  validity  within  different  levels  and  cores  of 
community  college  administration. 

From  the  conclusions  of  the  research  conducted  in  the 
present  study,  it  was  found  that  eight  of  the  personnel 
selection  methods  considered  are  valid  predictors  of 
administrative  achievement.  These  methods  are  the  overall 
assessment  rating  derived  from  assessment  center  processes, 
job-related  skills  indicators,  value  systems  assessments, 
personal  interviews,  aptitude  and  intelligence  measures,  peer 
ratings,  biographical  information,  and  psychological  attribute 
indicators . 

This  meta-analysis  has  provided  an  objective  view  of  the 
accumulated  findings  of  the  52  research  studies  involved.  The 
following  are  specific  guidelines  derived  from  the  meta- 
analysis of  the  research.  Taken  into  consideration  were  the 
legal  challenges  presented. 

1.  Set  up  an  assessment  center  for  the  identification  of 
persons  who  might  be  selected  as  community  college 
administrators.  The  assessment  center  overall  assessment 
rating  showed  the  highest  validity  of  all  the  selection 
methods.  To  offset  the  costs  involved,  use  the  center  also 
for  management  training.  Expected  outcomes  of  the  University 
of  California  assessment  center  method  are  the  improved 
performance  by  participants  in  their  present  jobs  and  improved 
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success  in  competing  for  entry-level  and  mid-level  management 
positions  in  the  university  system  (Brubaker,   1983)  . 

2.  Implement  a  statistical  combination  of  the  data  derived 
from  other  selection  processes.  Multiple  correlations  of 
scores  derived  from  several  selection  methods  increased  the 
validity  of  the  methods  to  predict  administrative  achievement 
(Howard,  1974) .  The  results  of  this  research  have  shown  that, 
where  selection  methods  were  used  independently  of  any 
assessment  center,  they  were  as  valid  as  the  same  methods  used 
within  an  assessment  center.  Thus,  the  assessment  center 
process  does  not  enhance  the  validity  of  the  specific 
selection  methods,  but  the  computational  synthesis  of  the 
combined  data  does  enhance  validity. 

3.  Write  a  detailed,  task-oriented  job  description  for  any 
administrative  position  to  be  filled.  This  process  is 
necessary  (a)  to  provide  selectors  with  specific  skills, 
traits,  and  capabilities  to  be  sought  within  all  selection 
processes  used,  and  (b)  to  satisfy  the  legal  constraints  on 
job-relatedness  of  personnel  selection  procedures.  Job- 
related  skills  indicators  ranked  second  in  overall  validity, 
also  having  a  confidence  range  above  the  significance  level. 

4.  Use  job-related  skills  indicators  wherever  possible 
within  the  selection  process.  As  indicated  by  the  results  of 
this  study,  a  job-related  selection  method  can  only  add  to  the 
validity  of  the  overall  selection  process.  The  process  of 
developing  job-related  skill  indicators  assures  that  the  legal 
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aspects  of  that  portion  of  the  job  search  have  been  met 
(Huegli  &  Eich,  1979). 

5.  Do  not  use  value  systems  assessments  in  the  selection 
process  for  administrators  in  public  community  colleges.  This 
selection  method  may  not  be  viewed  by  the  legal  system  as 
conforming  to  the  Higher  Education  Guidelines  for  job-related 
validity  (Wisner,  1975) .  However,  value  system  assessments 
ranked  third  in  validity,  showing  a  small  confidence  interval 
well  above  the  significance  level.  Within  the  course  of  the 
research  for  this  study,  the  researcher  noticed  specific  value 
systems  tests  which  showed  high  validity  in  predicting 
administrative  achievement. 

6.  Continue  to  use  personal  interviews  in  the  selection 
process.  The  personal  interview  almost  always  enhances  the 
validity  of  the  personnel  selection  process.  However,  more 
detailed  reporting  of  research  is  needed  to  determine  which 
specific  aspects  of  the  interview  method  enhance  its  ability 
to  validly  predict  future  administrative  achievement. 
Personal  interviews  ranked  fourth  in  overall  predictive 
validity.  This  finding  is  encouraging  since  the  personal 
interview  is  the  second  most  used  method  in  the  selection  of 
administrative  personnel. 

It  has  been  suggested  that  structured  interviews  and 
interviews  involving  more  than  one  rater  show  higher 
predictive  validities.  The  paucity  of  detailed  information 
concerning  these  aspects  of  the  research  reviewed  for  this 
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study  precluded  a  meta-analysis  for  the  effects  of  structured 
versus  nonstructured,  and  singly-rated  versus  multiply-rated, 
moderator  variables.  Thus,  research  within  this  selection 
method  is  needed  to  show  selectors  ways  to  boost  the  validity 
of  this  often-used  method. 

7.  Use  aptitude  and  intelligence  measures  to  raise  the 
caliber  of  people  being  admitted  to  educational  administration 
programs  in  colleges  and  universities,  and  thus,  eventually 
improve  the  labor  pool  from  which  administrators  are  drawn 
(Farguhar  &  Piele,  1972) .  In  this  study  it  was  found  that 
aptitude  and  intelligence  measures  ranked  fifth  from  the 
highest  in  predictive  validity  of  selection  methods.  The 
confidence  interval  was  almost  wholly  above  the  significance 
level.  Thus,  in  this  study,  it  was  shown  that  aptitude  and 
intelligence  measures  are  valid  predictors  of  future 
administrative  achievement.  The  problem  in  selection  of 
educational  administrators  is  more  a  result  of  admitting  weak 
applicants  into  the  selection  process  than  of  lacking  strong 
candidates.  Legality  may  be  an  issue  in  the  reguirement  of 
an  aptitude  test  as  a  selection  process  in  a  public  community 
college . 

8.  Suspect  the  validity  of  letters  of  recommendation.  The 
actual  validity  of  peer  ratings  used  in  real  life  situations 
may  well  be  less  than  is  shown  by  this  meta-analysis.  Peer 
ratings  ranked  below  the  median,  but  still  showed  significant 
validity    in    predicting    administrative    achievement.  The 
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subjective  reviews  of  the  research  on  peer  assessments 
(Haggerty,  1963;  Hollander,  1965;  Roadman,  1964)  lead  one  to 
believe  that  the  validity  of  this  method  might  be  higher  than 
it  actually  is. 

However,  attention  must  be  given  to  the  willingness  of 
participants  to  rate  each  other  accurately  when  they  know  full 
use  will  be  made  of  such  ratings  for  administrative  purposes 
(Roadman,  1964)  .  In  the  1980s,  many  persons  may  not  give 
truthful  opinions  of  another  person's  abilities,  or  lack 
thereof,  for  legal  reasons.  This  tendency  may  not  be  evident 
in  studies  conducted  for  research  purposes  only. 

9.  Conduct  detailed  research  within  the  selection  method 
of  biographical  information.  This  most  used  method  ranked 
seventh  in  predictive  validity.  The  findings  from  detailed 
study  of  this  method,  with  a  search  for  those  moderator 
variables  which  add  to  predictive  validity,  may  aid  in 
improving  the  use  of  this  method. 

10.  Avoid  using  psychological  attribute  indicators  as  a 
selection  method  for  the  reasons  that  (a)  although 
significant,  it  has  marginal  value  as  a  predictor  of 
administrative  achievement,  and  (b)  there  may  be  legal 
problems  with  testing  for  motivations  as  opposed  to  job- 
related  skills. 

The  measurement  of  psychological  attributes  as  a 
predictor  of  administrative  achievement  has  received  by  far 
the  most  attention  in  the  research.     Little  progress  has  been 
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made   in  applying  knowledge  toward   improving  the  predictive 

validity  of  this  method.     The  mean  correlation  of  this  method 

fell  near  the  significance  level.    Appleby  and  Nunnery  (1980) 

found  that  various  psychological  attributes  did  not  account 

for  differences  in  managerial  effectiveness  among  community 

college    department    heads.        Although    several    flaws  were 

acknowledged    in    the    study,    it    was    representative    of  the 

lackluster  contribution  of  this  selection  method  in  predicting 

achievement.        Perhaps    reconsideration    is    called    for  in 

researchers'    avid   pursuit    of    a    selection   method   that  has 

persistently  failed  to  enhance  selection  validity. 

Recommendations  for  Research 

More  research  should  be  conducted  on  personnel  selection 

within    education    in    general    and    in    community    colleges  in 

particular.         An     institutional     research     department  or 

organization  dealing  with  community  colleges  may  be  warranted. 

Only  3%  of  the  research  reviewed  for  this  study  was  conducted 

using  educational  administrators  as  subjects.     However,  46% 

of   the   researchers    involved   with   the    52    research  studies 

worked  for  universities  at  the  time  of  their  research.  As 

Richardson  (1987)  stated,  "university  professors  are  sometimes 

criticized  for  studying  everything  but  their  own  university 

settings"   (p.   39) . 

The  understaffed  and  inexperienced  colleges  of  the 
sixties  have  been  replaced  by  mature  institutions 
with  their  own  research  and  support  staffs.  The 
cutting-edge  community  college  of  this  decade 
possesses  more  resources  and  more  expertise  in 
virtually  every  line  or  administrative  activity  than 
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all      of      the      university      programs  combined. 
(Richardson,   1987,  p.  39) 

Hammons  and  Ivery  (1987)  found  significant  differences 
between  the  tasks  of  corporation  executives  and  community 
college  leaders.  Thus,  it  is  possible  that  the  results  of 
administrative  personnel  selection  research,  which  has  been 
done  predominately  within  corporations,  cannot  be  generalized 
to  the  field  of  education.  If  this  is  the  case,  such  research 
is  needed  within  the  field  of  higher  education. 

The  results  of  this  study  show  that  concurrent  validity 
studies  may  serve  as  well  as  predictive  studies  in  determining 
the  validity  of  administrative  personnel  selections  methods. 
Thus,  the  information  which  could  be  obtained  from  concurrent 
validity  studies  conducted  within  public  community  colleges 
may  add  to  our  knowledge  of  the  validities  of  administrative 
personnel  selection  in  the  near  term. 

Three  methodological  practices  are  suggested  as  a  result 
of  this  study:  (a)  use  a  combination  of  two  or  more  outcome 
measures  to  add  to  validity  in  measuring  administrative 
achievement,  (b)  plan  for  the  length  of  time  allowed  to  elapse 
between  collection  of  the  independent  and  dependent  variables 
to  be  at  least  three  years  but  not  greater  than  ten  years,  and 
(c)  report  details  of  research  conducted  so  that  researchers 
can  determine  what  moderator  variables  within  each  method 
contribute  to  validity. 
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Perhaps  most  important  of  all,  community  colleges 
and  university  graduate  programs  can  work  together 
to  promote  high-quality  research.  Such  research, 
to  be  worthy  of  doctoral  recognition,  should  conform 
to  rigorous  standards,  contribute  to  our  knowledge 
base  on  community  colleges,  and  have  an  impact  on 
professional  practice.      (Richardson,   1987,  p.  41) 

Summary  and  Conclusion 

The  two  administrative  personnel  selection  methods  which 

showed  high  predictive  validity  were  the  overall  assessment 

ratings    of    the    assessment    center   process    and  job-related 

skills  indicators.     The  first  of  these  methods  is  global  in 

aiding  in  the  selection  of  personnel  who  can  be  expected  to 

achieve  in  various  environments.     The  second  method  acquires 

its  validity  from  its  specificity  to  a  particular  set  of  job 

skills.     Guion  (1987)   stated  that 

if  your  purpose  is  to  hire  generally  good  people, 
a  global  criterion  is  useful.  If  you  aren't  very 
sure  what  distinguishes  good  from  better  employees, 
mix  up  a  little  of  every  kind  of  job  behavior  in 
developing  a  truly  global  criterion.  However,  if 
you  need  to  solve  a  very  specific  problem,  then  a 
more  specific  criterion  is  needed.  If  there  is  more 
than  one  specific  problem,  then  more  than  one 
specific  criterion  is  called  for.  But  in  most 
situations,  a  global  measure  will  serve  quite  well, 
(p.  205) 

The  personnel  selection  methods  which  are  currently  used 
for  the  selection  of  administrators  in  public  community 
colleges  showed  moderate  predictive  validity  in  this  study. 
Since  these  methods  likely  will  continue  to  be  used, 
suggestions  for  improvement  in  their  predictive  validities 
have  been  suggested. 
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This  objective  view  of  the  research,  applied  to  the 
selection  of  administrative  personnel  in  public  community 
colleges,  has  led  to  the  following  conclusions.  Use 
assessment  centers  in  higher  education  for  the  purposes  of 
both  personnel  selection  and  career  development.  Use  job- 
related  skills  indicators  to  aid  in  defining  the  position  to 
be  filled,  to  enhance  predictive  validity  in  selection,  and 
to  avoid  legal  problems.  Avoid  any  testing  that  is  not  job- 
related,  such  as  value  system  assessments  and  psychological 
attribute  indicators. 

Use  aptitude  and  intelligence  measures  to  improve  the 
labor  pool  feeding  educational  administrative  positions.  Look 
within  the  selection  methods  of  personal  interviews,  peer 
ratings,  and  biographical  information  for  the  moderator 
variables  which  increase  the  predictive  validities  of  these 
methods.  Use  computational  methods  to  combine  selection 
information  of  a  job  candidate  in  order  to  simulate  the 
validity  of  an  assessment  center  overall  assessment  rating. 


APPENDIX 
ABSTRACTS  OF  SELECTED  STUDIES 

1.  Bentz,  V.  J.  (1967).  Sears,  Roebuck  studies.  In  J.  P. 
Campbell,  M.  D.  Dunnette,  E.  E.  Lawler,  &  K.  E.  Weick,  Jr., 
Measuring  executive  effectiveness  (pp.  184-187) .  New  York: 
Appleton-Century-Crof ts . 

Purpose:     To  develop  a  psychological  testing  program  for  the 
prediction  of  executive  effectiveness  in  the  Sears 
organization. 

Type  organization:  Corporation. 

Subjects:     1,3  75  managers. 

Type  validity:  Predictive. 

Years  between  variables:     11  through  17. 

Assessment  center?:  Yes. 

Range  restriction:  Medium. 

Criterion  contamination?:     Assumed  yes. 

Results: 

a.  Predictor:     Aptitude  and  intelligence  measures. 
Criterion:     Promotion  rate. 

Correlation  coefficient  (r) :  0.21 

b.  Predictor:     Value  systems  assessments. 
Criterion:     Promotion  rate. 
Correlation  coefficient  (r) :  0.22 

c.  Predictor:     Psychological  attribute  indicators. 
Criterion:     Promotion  rate. 

Correlation  coefficient   (r) :  0.20 
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2.     Bray,   D.  W. ,   &  Grant,   D.   L.      (1966).     The  assessment 
center  in  the  measurement  of  potential  for  business 
management.     Psychological  Monographs.  80(17,  Whole  No. 
625)  . 

Purpose:     To  measure  personal  characteristics  hypothesized 
to  be  of  importance  either  in  developmental  change  in  early 
adulthood  or  success  in  business  management. 

Type  organization:  Corporation. 

Subjects:     422  male,  low-level  managers. 

Type  validity:  Predictive. 

Years  between  variables:  8 

Assessment  center?:  Yes. 

Range  restriction:  Medium. 

Criterion  contamination?:  No. 

Results: 

a.  Predictor:     Job-related  skills  indicators. 
Criterion:     Salary  level  attained. 
Correlation  coefficient   (r) :     0.26  (N=269) 

b.  Predictor:     Aptitude  and  intelligence  measures. 
Criterion:     Salary  level  attained. 
Correlation  coefficient   (r) :     0.27  (N=269) 

c.  Predictor:     Psychological  attribute  indicators. 
Criterion:     Salary  level  attained. 
Correlation  coefficient   fr) :     0.16  (N=269) 

d .  Predictor :     OAR . 

Criterion:     Salary  level  attained. 
Correlation  coefficient   (r) :     0.47  (N=203) 
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e .     Predictor :     OAR . 

Criterion:     Administrative  level  achieved. 

Correlation  coefficient   (r) :     0.57   (N=  422) 

3.     Campbell,  J.   P.,   Dunnette,  M.   D. ,   Lawler,   E.  E. ,  & 
Weick,  K.  E.     (1970) .     Managerial  behavior,  performance,  and 
effectiveness .     New  York:     McGraw  Hill. 

Purpose:     To  determine  how  employees  who  possess  the 
potential  to  be  successful  in  management  can  be  identified 
early  in  their  careers. 

Type  organization:  Corporation. 

Subjects:     443  mid-  to  high-level  managers. 

Type  validity:  Concurrent. 

Years  between  variables:  0 

Assessment  center?:  No. 

Range  restriction:  High. 

Criterion  contamination?:  N/A. 

Results: 

a.  Predictor:     Aptitude  and  intelligence  measures. 
Criterion:  Combination. 

Correlation  coefficient  (r) :  0.16 

b.  Predictor:     Psychological  attribute  indicators. 
Criterion:  Combination. 

Correlation  coefficient  (r) :  0.08 

c.  Predictor:  Self-appraisals. 
Criterion:  Combination. 
Correlation  coefficient   (r) :  0.24 
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d.     Predictor:     Job-related  skills  indicators. 


Criterion: 

Combination. 
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Personal  interviews. 

Criterion: 

Combination . 

Correlation 

coefficient 

(r)  : 

0. 

20 

rreuictor: 

Biographical 

information 

Criterion: 

Combination. 

Correlation 

coefficient 

fr)  : 

0. 

57 

Predictor: 

OAR 

Criterion: 

Combination. 

Correlation 

coefficient 

fr)  : 

0. 

30 

4.     Carleton,   F.  O.    (1970).     Relationships  between  follow-up 
evaluations  and  information  developed  in  a  management 
assessment  center.     Proceedings  of  the  78th  Annual 
Convention  of  the  American  Psychological  Association ,  pp . 
565-568. 

Purpose:     To  determine  if  test  and  rating  data  generated  in 
a  multiple-technique  assessment  center  hold  up  well  as 
predictors  of  behavioral  ratings  made  several  years  later. 

Type  organization:  Corporation. 

Subjects:     122  managers. 

Type  validity:  Predictive. 

Years  between  variables:     2.5  to  5 

Assessment  center?:  Yes. 

Range  restriction:  Medium. 

Criterion  contamination?:  Yes. 

Results : 

a.     Predictor:     Personal  interviews. 
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Criterion :     Supervisor  ratings. 
Correlation  coefficient  (r)  :  0.23 
b .     Predictor  (selection  method) :  OAR 
Criterion:     Supervisor  ratings. 
Correlation  coefficient   fr) :  0.65 

5.  Cummin,   P.   C.    (1967).     TAT  correlates  of  executive 
performance.     Journal  of  Applied  Psychology.   51.  78-81. 

Purpose:     To  determine  if  successful  executives  rate  high  in 
the  psychological  attributes  of  achievement,  power,  and 
autonomy,  and  less  successful  executives  rate  high  in 
affiliation,  aggression,  and  deference. 

Type  organization:  Corporation. 

Subjects:     52  mid-  and  high-level  managers. 

Type  validity:  Concurrent. 

Years  between  variables:  0 

Assessment  center?:  No. 

Range  restriction:  High. 

Criterion  contamination?:  N/A. 

Results : 

Predictor:     Psychological  attribute  indicators. 
Criterion:     Salary  level  attained. 
Correlation  coefficient  (r) :  0.21 

6.  Dicken,  C. ,   &  Black,  J.     (1965).     Predictive  validity  of 
psychometric  evaluation  of  supervisors.     Journal  of  Applied 
Psychology.   49,  34-47. 

Purpose:     To  study  the  validity  of  clinical  interpretations 
of  an  objective  test  battery  in  two  different  industrial 
settings . 

Type  organization:  Corporation. 

Subjects:     2  samples:     (a)   31  male,  low-level  managers,  (b) 
2  6  male,   low-level  managers. 


Type  validity:  Predictive. 

Years  between  variables:      (a)   3.5,    (b)  7 

Assessment  center?:  No. 

Range  restriction:  Medium. 

Criterion  contamination?:  Yes. 

Results: 

a.  Predictor:     Aptitude  and  intelligence  measures 
Criterion:     Salary  level  attained. 
Correlation  coefficient   (r) :     0.41  (N=31) 

b.  Predictor:     Psychological  attribute  indicators 
Criterion:     Salary  level  attained. 
Correlation  coefficient   (r) :     0.15  (N=31) 

c.  Predictor:     Job-related  skills  indicators. 
Criterion:     Salary  level  attained. 
Correlation  coefficient   (r) :     0.35  (N=31) 

d.  Predictor:     Aptitude  and  intelligence  measures 
Criterion:     Promotion  rate. 

Correlation  coefficient   fr) :     0.21  (N=31) 

e.  Predictor:     Psychological  attribute  indicators. 
Criterion:     Promotion  rate. 

Correlation  coefficient  (r) :     0.11  (N=31) 

f.  Predictor:     Job-related  skills  indicators. 
Criterion:     Promotion  rate. 
Correlation  coefficient   fr) :     0.11  (N=31) 

g.  Predictor:     Aptitude  and  intelligence  measures. 
Criterion:     Salary  level  attained. 
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Correlation  coefficient   (r);      0.29  (N=26) 

h.  Predictor:  Psychological  attribute  indicators. 
Criterion:  Salary  level  attained. 
Correlation  coefficient  (r) :     0.15  (N=26) 

i.  Predictor:  Aptitude  and  intelligence  measures. 
Criterion:  Promotion  rate. 

Correlation  coefficient  (r) :     0.18  (N=26) 

j.     Predictor:  Psychological  attribute  indicators. 

Criterion:  Promotion  rate. 

Correlation  coefficient   (r) :     0.20  (N=26) 

7.  Edel,  E.  G.  (1968).  "Need  for  success"  as  a  predictor 
of  managerial  performance.  Personnel  Psychology,  21,  231- 
240. 

Purpose:     To  determine  the  concurrent  validity  of  a  conative 
measure  which  purports  to  assess  "need  for  success"  by 
measuring  perceptual  distortion  to  "loaded"  stimuli  in  an 
indirect  manner     (Edel,   1968,  p.   232) . 

Type  organization:  Corporation. 

Subjects:     232  first-level  managers. 

Type  validity:  Concurrent. 

Years  between  variables:  0 

Assessment  center?:  No. 

Range  restriction:  High. 

Criterion  contamination?:  N/A 

Results: 

Predictor:     Psychological  attribute  indicators. 
Criterion:     Supervisor  ratings. 
Correlation  coefficient   (r) :  0.43 
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8.  England,  G.  W. ,  &  Lee,  R.  (1974).  The  relationship 
between  managerial  values  and  managerial  success  in  the 
United  States,  Japan,  India,  and  Australia.  Journal  of 
Applied  Psychology.   39,  411-419. 

Purpose:     To  investigate  the  relationship  between  values  of 
managers  and  their  success  as  managers  in  four  countries. 

Type  organization:  Corporation. 

Subjects:     878  high-level  managers. 

Type  validity:  Concurrent. 

Years  between  variables:  0 

Assessment  center?:  No. 

Range  restriction:  High. 

Criterion  contamination?:  N/A 

Results: 

Predictor:     Value  systems  assessments. 
Criterion:     Salary  level  attained. 
Correlation  coefficient   (r) :  0.32 

9.  Finley,  R.  M. ,  Jr.      (1970).     Predictions  from  projective 
tests  given  in  a  management  assessment  center.     In  A.  Howard 
(1974).     An  assessment  of  assessment  centers,  Academy  of 
Management  Journal,   17,  123. 

Purpose:     To  study  the  predictive  validity  of  projective 
tests  in  a  management  assessment  center. 

Type  organization:  Corporation. 

Subjects:     109  subjects. 

Type  validity:  Predictive. 

Years  between  variables:     .75  through  5 

Assessment  center?:  Yes. 

Range  restriction:     Assumed  medium. 

Criterion  contamination?:  Yes. 
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Results: 

Predictor:  OAR 

Criterion:     Supervisor  ratings. 
Correlation  coefficient   (r) :  0.33 

10.  Fleishman,   E.  A.,   &  Peters,   D.  R.  (1962). 
Interpersonal  values,  leadership  attitudes,  and  managerial 
"success".     Personnel  Psychology.   15,  127-143. 

Purpose:     To  study  the  interrelationships  among  criteria  of 
leadership  effectiveness,  value  dimensions,  and  leader 
behavior  and  attitudes. 

Type  organization:  Corporation. 

Subjects:     2  5  Managers 

Type  validity :     Concurrent . 

Years  between  variables:  0 

Assessment  center?:  No. 

Range  restriction:  High. 

Criterion  contamination?:  N/A 

Results : 

Predictor:     Psychological  attribute  indicators. 
Criterion:     Supervisor  ratings. 
Correlation  coefficient  (r) :  0.19 

11.  Gantz,  B.  S.,  Erickson,  C.  0.,   &  Stephenson,  R.  W. 
Some  determinants  of  promotion  in  a  research  and  development 
population.     In  J.   B.  Miner,   &  M.  G.  Miner  (1977) . 
Motivation  to  manage:     A  ten  year  update  on  the  "studies  in 
management  education"  research   (pp.   18-22) .  Atlanta: 
Organizational  Measurement  Systems  Press. 

Purpose:     To  gain  more  theoretical  understanding  of  why  some 
persons  in  a  research  and  development  population  achieve 
promotion  into  managerial  positions  and  others  do  not. 

Type  organization:  Corporation. 


Subjects:     117  engineers  and  research  scientists. 


Type  validity:  Concurrent. 
Years  between  variables:  0 
Assessment  center?:  No. 
Range  restriction:  High. 
Criterion  contamination?:  N/A 
Results : 

a.  Predictor:     Aptitude  and  intelligence  measures 
Criterion:     Promotion  rate. 

Correlation  coefficient   fr) :  0.06 

b.  Predictor:     Aptitude  and  intelligence  measures 
Criterion:     Supervisor  ratings. 
Correlation  coefficient  (r) :  0.07 

c.  Predictor:     Psychological  attribute  indicators 
Criterion:     Promotion  rate. 

Correlation  coefficient   (r) :  0.07 

d.  Predictor:     Psychological  attribute  indicators 
Criterion:     Supervisor  ratings. 
Correlation  coefficient  (r) :  0.05 

e.  Predictor:     Biographical  information. 
Criterion:     Promotion  rate. 
Correlation  coefficient   (r) :  0.17 

f.  Predictor:     Biographical  information. 
Criterion:     Supervisor  ratings. 
Correlation  coefficient   (r) :  0.15 

g.  Predictor:     Peer  ratings. 
Criterion:     Promotion  rate. 
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Correlation  coefficient  fr) :  0.26. 

12.  Gantz ,   B.  S.,  Erickson,   C.  O. ,   &  Stephenson,  R.  W.  The 
prediction  of  managerial  and  research  success.     In  J.  B. 
Miner,   &  M.  G.  Miner  (1977) .     Motivation  to  manage:     A  ten 
year  update  on  the  "studies  in  management  education" 
research  (pp.   28-34) .     Atlanta:     Organizational  Measurement 
Systems  Press. 

Purpose:     To  determine  whether  motives  measured  by  a 
psychological  attribute  indicators  are  the  cause  of 
managerial  success. 

Type  organization:  Corporation. 

Subjects:     49  male  research  scientists. 

Type  validity:  Predictive. 

Years  between  variables:  5.3 

Assessment  center?:  No. 

Range  restriction :     Low . 

Criterion  contamination?:     Assumed  yes. 

Results : 

Predictor:     Psychological  attribute  indicators. 
Criterion:     Promotion  rate. 
Correlation  coefficient   (r) :  0.25 

13.  Gantz,   B.  S.,  Erickson,   C.  O. ,   &  Stephenson,  R.  W.  The 
marketing  department  follow-up  study.     In  J.  B.  Miner,   &  M. 
G.  Miner  (1977) .     Motivation  to  manage:     A  ten  year  update 
on  the  "studies  in  management  education"  research     (pp.  28- 
34) .     Atlanta:     Organizational  Measurement  Systems  Press. 


Purpose:  To  determine  whether  motives  measured  by  a 
psychological  attribute  indicators  are  the  cause  of 
managerial  success. 

Type  organization:  Corporation. 

Subjects:     81  top  salesmen  and  marketing  managers. 
Type  validity:  Predictive. 
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Years  between  variables;  4 
Assessment  center?:  No. 
Range  restriction:  Medium. 
Criterion  contamination?:  No. 
Results: 

Predictor:     Psychological  attribute  indicators. 
Criterion:     Promotion  rate. 
Correlation  coefficient   (r):  0.39 

14.  Ghiselli,   E.  E.    (1966).     The  validity  of  a  personnel 
interview.     Personnel  Psychology,   19,  389-394. 

Purpose :  To  examine  the  proposition  that  superficial  data 
pertaining  to  what  an  individual  has  done  has  little  power 
in  predicting  job  "success". 

Type  organization:  Corporation. 

Subjects:     275  applicants  for  low-level  executive  positions. 

Type  validity:  Predictive. 

Years  between  variables:  3 

Assessment  center?:  No. 

Range  restriction :     Low . 

Criterion  contamination?:  Yes. 

Results : 

Predictor:     Personal  interviews. 

Criterion:     Number  of  years  serving  in  an 
administrative  position  within  the  same 
organization. 

Correlation  coefficient  (r) :  0.35 

15.  Ghiselli,  E.  E.     (1968).     Some  motivational  factors  in 
the  success  of  managers.     Personnel  Psychology.   21,  431-440. 


Purpose:     To  study  the  relationship  between  the  strength 
desire  for  self-actualization  and  performance  among 
managers. 

Type  organization:  Corporation. 
Subjects:     233  mid-level  male  managers. 
Type  validity:  Concurrent. 
Years  between  variables:  0 
Assessment  center?:  No. 
Range  restriction:  High. 
Criterion  contamination?:  N/A 
Results : 

a.  Predictor:     Psychological  attribute  indicators. 
Criterion:     Supervisor  ratings. 
Correlation  coefficient  (r) :     0.22  (N=89) 

b.  Predictor:     Psychological  attribute  indicators. 
Criterion:     Supervisor  ratings. 
Correlation  coefficient   fr) :     0.30  (N=21) 

c     Predictor:     Psychological  attribute  indicators. 
Criterion:     Supervisor  ratings. 
Correlation  coefficient   (r) :     0.47  (N=20) 

d.  Predictor:     Psychological  attribute  indicators. 
Criterion:     Supervisor  ratings. 
Correlation  coefficient   (r) :     0.22  (N=22) 

e.  Predictor:     Psychological  attribute  indicators. 

Criterion:     Number  of  years  serving  in  an 
administrative  position  within  the  same 
organization. 

Correlation  coefficient  fr) :     0.19  (N=81) 
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16.  Goodstein,   L.   D. ,   &  Schrader,  W.  J.      (1963).  An 
empirically-derived  managerial  key  for  the  California 
Psychological  Inventory.     Journal  of  Applied  Psychology,  47 
42-45. 

Purpose;     To  determine  the  usefulness  of  a  specific 
indicator  of  psychological  attributes  in  identifying  those 
personality  characteristics  associated  with  managerial  and 
supervisory  success  in  a  large  industrial  organization. 

Type  organization:  Military. 

Subjects:     603  mid-  and  top-level  male,   civilian  managers. 

Type  validity:  Concurrent. 

Years  between  variables:  0 

Assessment  center?:  No. 

Range  restriction:  High. 

Criterion  contamination?:  N/A 

Results : 

Predictor:     Psychological  attribute  indicators. 
Criterion:     Administrative  level  achieved. 
Correlation  coefficient   (r) :  0.23 

17.  Grant,   D.   L. ,   &  Bray,   D.  W.      (1969).     Contributions  of 
the  interview  to  assessment  of  managerial  potential. 
Journal  of  Applied  Psychology,   53,  24-34. 

Purpose:     To  present  information  on  the  contributions  made 
by  the  interview  to  the  assessment  center  process  and  to 
study  the  relationships  of  interview  variables  to  the 
progress  criterion. 

Type  organization:  Corporation. 

Subjects:     203  low-level  male  managers. 

Type  validity:  Predictive. 

Years  between  variables:  9 

Assessment  center?:  Yes. 

Range  restriction:  Medium. 
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Criterion  contamination?:  No. 
Results : 

Predictor:     Personal  interviews. 
Criterion:     Salary  level  attained. 
Correlation  coefficient  (r) :  0.23 

18.  Grant,   D.   L. ,  Katkovsky,  W. ,   &  Bray,   D.  W.  (1967). 
Contributions  of  projective  techniques  to  assessment  of 
management  potential.     Journal  of  Applied  Psychology.  51, 
226-232 . 

Purpose :     To  present  data  from  the  Bell  System  Management 
Progress  Study  (Bray,   1964)   on  the  relationships  between 
projective  techniques  and  managerial  achievement. 

Type  organization:  Corporation. 

Subjects:     201  male  low-level  managers. 

Type  validity:  Predictive. 

Years  between  variables:  8 

Assessment  center?:  Yes. 

Range  restriction:  Medium. 

Criterion  contamination?:  No. 

Results : 

Predictor:     Psychological  attribute  indicators. 
Criterion:     Salary  level  attained. 
Correlation  coefficient  (r) :  0.18 

19.  Grimsley,  G. ,   &  Jarrett,  G.   F.      (1975).     The  relation 
of  past  managerial  achievement  to  test  measures  obtained  in 
the  employment  situation:     Methodology  and  results-II. 
Personnel  Psychology.   2_8,  215-231. 

Purpose:     To  test  the  effectiveness  of  a  particular 
methodological  approach  which  can  be  used  in  analyzing  data 
gathered  in  the  process  of  assessing  managerial  applicants 
in  the  employment  situation. 
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Type  organization:  Corporation. 

Subjects:     2  samples  of  100  mid-  and  high-level  managers 
each. 

Type  validity:  Concurrent. 
Years  between  variables:  0 
Assessment  center?:  No. 
Range  restriction:  High. 
Criterion  contamination?:  N/A 
Results: 

a.  Predictor:     Aptitude  and  intelligence  measures. 
Criterion:     Administrative  level  achieved. 
Correlation  coefficient   fr) :  0.36 

b.  Predictor:     Psychological  attribute  indicators. 
Criterion:     Administrative  level  achieved. 
Correlation  coefficient  (r) :  0.17 

c.  Predictor:     Value  systems  assessments. 
Criterion:     Administrative  level  achieved. 
Correlation  coefficient  (r) :  0.11 

d.  Predictor:     Aptitude  and  intelligence  measures. 
Criterion:     Administrative  level  achieved. 
Correlation  coefficient  fr) :  0.35 

e.  Predictor:     Psychological  attribute  indicators. 
Criterion:     Administrative  level  achieved. 
Correlation  coefficient   fr) :  0.17 

f.  Predictor:     Value  systems  assessments. 
Criterion:     Administrative  level  achieved. 
Correlation  coefficient   (r) :  0.10 
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20.  Haggerty,  H.  R.    (1963) .     Status  report  on  research  for 
the  U.S.  Military  Academy   (Cadet  Leaders  Task) .  Washington, 
DC:     U.S.  Army  Personnel  Research  Office. 

Purpose :     To  study  the  relationships  between  certain 
predictor  variables  and  achievement  of  Army  officers. 

Type  organization:  Military. 

Subjects:     2  samples  of  78  and  420  U.S.  Military  Academy 
Cadets . 

Type  validity:  Predictive. 
Years  between  variables:  Unknown. 
Assessment  center?:  No. 
Range  restriction :     Low . 
Criterion  contamination?:  Yes. 
Results : 

a.  Predictor:     Peer  ratings. 
Criterion:     Supervisor  ratings. 
Correlation  coefficient   (r) :     0.53  (N=78) 

b.  Predictor:     Peer  ratings. 
Criterion:  Combination. 

Correlation  coefficient   (r) :     0.33  (N=420) 

c.  Predictor:     Biographical  information. 
Criterion:  Combination. 

Correlation  coefficient   (r) :     0.18  (N=78) 

21.  Harrell,  T.  W. ,   &  Harrell,  M.   S.      (1974).  Predictors 
of  management  success   (Technical  Report  No.   3).  Arlington, 
VA:     Office  of  Naval  Research. 

Purpose:     To  determine  predictors  of  management  success. 
Type  organization:  Corporation. 
Subjects:     2  05  MBA  graduates. 
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Type  validity:  Predictive. 
Years  between  variables:  10 
Assessment  center?:  No. 
Range  restriction :     Low . 
Criterion  contamination?:  Yes. 
Results : 

a.  Predictor:     Psychological  attribute  indicators. 
Criterion:     Salary  level  attained. 
Correlation  coefficient  fr) :  0.19 

b.  Predictor:     Biographical  information. 

Criterion:     Salary  level  attained. 

Correlation  coefficient   fr) :  0.28 

22.     Heneman,  H.  G.   Ill     (1974).     Comparisons  of  self-  and 
superior  ratings  of  managerial  performance.     Journal  of 
Applied  Psychology.   59,  638-642. 

Purpose:     To  identify  broad  dimensions  of  managerial 
performance,  obtain  self-  and  superior  performance  ratings, 
and  assess  these  ratings  in  terms  of  leniency,  restriction 
of  range,  halo,  and  convergent-discriminant  validity. 

Type  organization:  Combination. 
Subjects:     102  MBA  graduates. 
Type  validity:  Concurrent. 
Years  between  variables:  0 
Assessment  center?:  No. 
Range  restriction:  High. 
Criterion  contamination?:  N/A 
Results : 

Predictor:  Self-appraisals. 
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Criterion:     Supervisor  ratings. 
Correlation  coefficient   (r) :  0.22 

23.  Hicks,  J.  A.,   &  Stone,  J.   B.      (1962).  The 
identification  of  traits  related  to  managerial  success. 
Journal  of  Applied  Psychology.   46,  428-432. 

Purpose:     To  determine  if  a  broad  battery  of  tests  covering 
aptitudes,  temperament,  and  creativity  could  be  used  to 
identify  certain  basic  characteristics  for  selection, 
promotion,  and  training  purposes. 

Type  organization:  Corporation. 

Subjects:     76  low-level  supervisors. 

Type  validity:  Concurrent. 

Years  between  variables:  0 

Assessment  center?:  No. 

Range  restriction:  High. 

Criterion  contamination?:  N/A 

Results : 

a.  Predictor:     Psychological  attribute  indicators. 
Criterion:     Supervisor  ratings. 
Correlation  coefficient  fr) :  0.12 

b.  Predictor:     Aptitude  and  intelligence  measures. 
Criterion:     Supervisor  ratings. 
Correlation  coefficient  (r) :  0.17 

24.  Hinrichs,  J.  R.      (1978).     An  eight-year  follow-up  of  a 
management  assessment  center.     Journal  of  Applied 
Psychology.   63,  596-601. 

Purpose:     To  evaluate  the  predictive  validity  of  the 
assessment  center  process  and  to  compare  the  predictive 
accuracy  of  the  assessment  center  with  the  naturalistic 
management  evaluation. 

Type  organization:  Corporation. 
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Subjects:     30  male  nonmanagement  and  low-level  management 
personnel . 

Type  validity:  Predictive. 
Years  between  variables:  8 
Assessment  center?:  Yes. 
Range  restriction:  Medium. 
Criterion  contamination?:  No. 
Results : 

a.  Predictor:     Value  systems  assessments. 
Criterion:     Administrative  level  achieved. 
Correlation  coefficient  fr) :  0.26 

b.  Predictor:  Self-appraisals. 
Criterion:     Administrative  level  achieved. 
Correlation  coefficient   (r) :  0.26 

c.  Predictor:     Aptitude  and  intelligence  measures. 
Criterion:     Administrative  level  achieved. 
Correlation  coefficient   (r) :  0.21 

d .  Predictor :  OAR 

Criterion:     Administrative  level  achieved. 
Correlation  coefficient   (r) :  0.46 

e.  Predictor:     Biographical  information. 

Criterion:     Administrative  level  achieved. 

Correlation  coefficient   (r) :  0.55 

25.     Hollander,  E.  P.     (1965).     Validity  of  peer  nominations 
in  predicting  a  distant  performance  criterion.     Journal  of 
Applied  Psychology,   49,  434-438. 

Purpose:     To  determine  the  validity  of  peer  ratings  in 
predicting  future  administrative  achievement. 
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Type  organization:  Military. 

Subjects:     639  Naval  Officer  Candidate  School  students. 

Type  validity:  Predictive. 

Years  between  variables:  3 

Assessment  center?:  No. 

Range  restriction:  Low 

Criterion  contamination?:     Assumed  yes. 

Results : 

Predictor:     Peer  ratings. 

Criterion:     Supervisor  ratings. 

Correlation  coefficient  (r) :  0.35 

26.     Huck,  J.  R. ,   &  Bray,   D.  W.      (1976).  Management 
assessment  center  evaluations  and  subseguent  job  performance 
of  white  and  black  females.     Personnel  Psychology r   29,  13- 
30. 

Purpose:     To  test  the  validity  of  an  assessment  center 
process  on  a  population  different  from  the  population  of  the 
Management  Progress  Study. 

Type  organization:  Corporation. 

Subjects:     126  nonmanagement  women. 

Type  validity:  Predictive. 

Years  between  variables:     1.2  through  5.8 

Assessment  center?:  Yes. 

Range  restriction :     Low . 

Criterion  contamination?:  Yes. 

Results : 

a.     Predictor:     Aptitude  and  intelligence  measures. 
Criterion:     Supervisor  ratings. 
Correlation  coefficient   (r) :  0.15 
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b.  Predictor:  Job-related  skills  indicators. 
Criterion;  Supervisor  ratings. 
Correlation  coefficient  (r) :  0.27 

c .  Predictor :  OAR 

Criterion:  Supervisor  ratings. 

Correlation  coefficient  (r) :  0.38 

27.  Kotula,   L.  J.,   &  Haggerty,  H.  R.      (1966).     Research  in 
the  selection  of  officer  candidates  and  cadets.  Washington, 
DC:     U.S.  Army  Personnel  Research  Office. 

Purpose:     To  determine  the  predictive  validity  of  the 
personal  history  blank. 

Type  organization:  Military. 

Subjects:     426  Army  officers. 

Type  validity:  Concurrent. 

Years  between  variables:  0 

Assessment  center?:  No. 

Range  restriction:  High. 

Criterion  contamination?:  N/A 

Results : 

Predictor:     Biographical  information. 
Criterion:     Supervisor  ratings. 
Correlation  coefficient   (r)  :  0.17 

28.  Kraut,  A.   I.      (1969).     Intellectual  ability  and 
promotional  success  among  high  level  managers.  Personnel 
Psychology.   22,  281-290. 

Purpose:     To  study  the  relationship  of  high  level  managers' 
promotional  success  to  two  tests  of  intellectual 
functioning. 

Type  organization:  Corporation. 
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Subjects:     365  mid-  and  high-level  managers. 

Type  validity:  Predictive. 

Years  between  variables:     4  through  7 

Assessment  center?:  No. 

Range  restriction:  High. 

Criterion  contamination?:  Yes. 

Results : 

Predictor:     Aptitude  and  intelligence  measures. 
Criterion:     Promotion  rate. 
Correlation  coefficient  (r) :     . 02 

29.  Kraut,  A.   I.,   &  Scott,  G.  J.      (1972).     Validity  of  an 
operational  management  assessment  program.     Journal  of 
Applied  Psychology.   56,  124-129. 

Purpose:     To  study  the  predictive  validity  of  an  assessment 
center. 

Type  organization:  Corporation. 
Subjects:     4  32  nonmanagement  employees. 
Type  validity:  Predictive. 
Years  between  variables:     1  through  5 
Assessment  center?:  Yes. 
Range  restriction :     Low . 
Criterion  contamination?:  Yes. 
Results : 

Predictor :  OAR 

Criterion:     Administrative  level  achieved. 
Correlation  coefficient  (r) :  0.25 

30.  Lawler,  E.  E.  Ill  (1967).  The  multitrait-multirater 
approach  to  measuring  managerial  job  performance.  Journal 
of  Applied  Psychol ogyr    51.  369-381. 
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Purpose:     To  study  the  advantages  of  the  multitrait- 
multirater  approach  to  measuring  managerial  job  performance. 

Type  organization:  Corporation. 

Subjects:     113  mid-  to  high-level  managers. 

Type  validity:  Concurrent. 

Years  between  variables:  0 

Assessment  center? :  No. 

Range  restriction:  High. 

Criterion  contamination?:  N/A 

Results : 

a.  Predictor:  Self-appraisals. 
Criterion:     Supervisor  ratings. 

Correlation  coefficient   (r) :     Approximately  0.07 

b.  Predictor:     Peer  ratings. 

Criterion:     Supervisor  ratings. 

Correlation  coefficient   (r) :  0.43 

31.     Mayfield,   E.  C.      (1970).     Management  selection:  Buddy 
nominations  revisited.     Personnel  Psychology ,   23 ,  377-391. 

Purpose:     To  investigate  the  value  of  the  buddy  nomination 
procedure  in  the  selection  of  assistant  managers. 

Type  organization:  Corporation. 

Subjects:     154  real  estate  agents. 

Type  validity:  Predictive. 

Years  between  variables:     2 . 5 

Assessment  center? :  No. 

Range  restriction :     Low . 

Criterion  contamination?:  No. 
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Results : 

Predictor:     Peer  ratings. 
Criterion:     Supervisor  ratings. 
Correlation  coefficient  (r) :  0.23 

32.  McClelland,   D.   C. ,   &  Boyatzis,  R.  E.  (1982). 
Leadership  motive  pattern  and  long-term  success  in 
management.     Journal  of  Applied  Psychology.   67,  737-743. 
Purpose:     To  determine  whether  specific  personality 
characteristics  relate  to  success  in  management. 

Type  organization:  Corporation. 

Subjects:     237  male,  entry-level  managers. 

Type  validity:  Predictive. 

Years  between  variables:  16 

Assessment  center?:  Yes. 

Range  restriction:  Medium. 

Criterion  contamination?:  No. 

Results : 

Predictor:     Psychological  attribute  indicators. 
Criterion:     Administrative  level  achieved. 
Correlation  coefficient   (r) :  0.33 

33.  Meyer,  H.  H.     (1970).     The  validity  of  the  in-basket 
test  as  a  measure  of  managerial  performance.  Personnel 
Psychology.   23 ,  297-307. 

Purpose:     To  determine  to  what  degree  performance  on  an  in- 
basket  test  actually  does  correspond  to  observed  performance 
of  an  actual  managerial  job. 

Type  organization:  Corporation. 

Subjects:     81  male,   low-level  managers. 

Type  validity:  Concurrent. 

Years  between  variables:  0 
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Assessment  center?:  No. 
Range  restriction:  High. 
Criterion  contamination?:  N/A 
Results : 

Predictor:     Job-related  skills  indicators. 

Criterion:     Supervisor  ratings. 

Correlation  coefficient  (r) :  0.29 

34.     Miner,  J.   B. ,   &  Miner,  M.  G.      (1977).     Motivation  to 
manage:     A  ten  year  update  on  the  "Studies  in  Management 
Education"  research.     Atlanta:     Organizational  Measurement 
Systems  Press. 

Purpose:     To  study  the  relationship  between  two 
psychological  attribute  measures  and  indexes  of  management 
success . 

Type  organization:  Corporation. 
Subjects:     101  high-level  managers. 
Type  validity :     Concurrent . 
Years  between  variables:  0 
Assessment  center?:  No. 
Range  restriction:  High. 
Criterion  contamination?:  N/A 
Results : 

a.  Predictor:     Psychological  attribute  indicators. 
Criterion:     Administrative  level  achieved. 
Correlation  coefficient   (r) :  0.10 

b.  Predictor:     Psychological  attribute  indicators. 
Criterion:     Salary  level  attained. 
Correlation  coefficient  (r)  :  0.13 
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35.  Miner,  J.   B. ,   &  Miner,  M.  G.      (1977).     Motivation  to 
manage:     A  ten  year  update  on  the  "Studies  in  Management 
Education"  research.     Atlanta:     Organizational  Measurement 
Systems  Press. 

Purpose:     To  study  the  relationship  between  psychological 
attribute  indicators  and  professional  and  administrative 
success  in  educational  administration. 

Type  organization:  Educational. 

Subjects :     49  university  administrators  and  faculty  members. 

Type  validity:  Concurrent. 
Years  between  variables:  0 
Assessment  center?:  No. 
Range  restriction:  High. 
Criterion  contamination?:  N/A 
Results : 

Predictor:     Psychological  attribute  indicators. 
Criterion:     Administrative  level  achieved. 
Correlation  coefficient  (r) :  0.10 

36.  Miner,  J.   B. ,   &  Miner,  M.  G.      (1977).     Motivation  to 
manage:     A  ten  year  update  on  the  "Studies  in  Management 
Education"  research.     Atlanta:     Organizational  Measurement 
Systems  Press. 

Purpose:     To  determine  the  relationships  between  motivation 
to  manage  and  managerial  success  in  educational 
organizations. 

Type  organization:  Educational. 

Subjects:     219  mid-to  high-level  educational  administrators. 
Type  validity:  Concurrent. 
Years  between  variables:  0 
Assessment  center?:  No. 
Range  restriction:  High. 
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Criterion  contamination?;  N/A 
Results: 

Predictor:     Psychological  attribute  indicators. 

Criterion:     Salary  level  attained. 

Correlation  coefficient   (r) :  0.05 

37.     Miner,  J.   B. ,   &  Miner,  M.  G.      (1977).     Motivation  to 

manage:  A  ten  year  update  on  the  "Studies  in  Management 

Education"  research.  Atlanta:  Organizational  Measurement 
Systems  Press. 

Purpose:  To  test  the  hypothesis  that  sex  differences  in 
motivation  to  manage  do  exist  among  practicing  managers. 

Type  organization:  Educational. 

Subjects:     219  male  and  female  mid-  to  high-level 
educational  administrators. 

Type  validity:  Concurrent. 

Years  between  variables:  0 

Assessment  center?:  No. 

Range  restriction:  High. 

Criterion  contamination?:  N/A 

Results : 


Predictor:     Psychological  attribute  indicators. 

Criterion:     Supervisor  ratings. 

Correlation  coefficient  (r) :  0.18 

38.  Mitchel,  J.  o.  (1975).  Assessment  center  validity:  A 
longitudinal  study.     Journal  of  Applied  Psychology,   60,  573- 


Purpose:     To  examine  the  predictive  validity  of  an 
assessmment  center  in  a  developmental  paradigm,   and  to 
observe  changes  in  the  validity  coefficients  resulting  from 
measuring  a  criterion  of  salary  growth  at  different  points 
m  time. 
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Type  organization:  Corporation. 
Subjects:     254  low-level  managers. 
Type  validity:  Predictive. 
Years  between  variables:  5 
Assessment  center?:  Yes. 
Range  restriction:  Medium. 
Criterion  contamination?:  Yes. 
Results : 

a.  Predictor:     Peer  ratings. 
Criterion:     Salary  level  attained. 
Correlation  coefficient   (r) :  0.23 

b .  Predictor :  OAR 

Criterion:     Salary  level  attained. 

Correlation  coefficient   (r) :  0.14 

39.  Moses,  J.  L.  (1972).  Assessment  center  performance 
and  management  progress.  Studies  in  Personnel  Psychology 
4(1),   7-12.  "   ax' 

Purpose:     To  study  the  relationship  between  assessment 
center  processes  and  managerial  achievement. 

Type  organization:  Corporation. 

Subjects:     8,885  nonmanagement  males. 

Type  validity:  Predictive. 

Years  between  variables:  7 

Assessment  center?:  Yes. 

Range  restriction:  Low. 

Criterion  contamination?:  Yes. 

Results : 

a.     Predictor:     Personal  interviews. 
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Criterion:     Administrative  level  achieved. 
Correlation  coefficient  (r) :  0.27 

b.  Predictor:     Aptitude  and  intelligence  measures. 
Criterion:     Administrative  level  achieved. 
Correlation  coefficient  (r) :  0.21 

c.  Predictor:  Job-related  skills  indicators. 
Criterion:  Administrative  level  achieved. 
Correlation  coefficient  fr) :  0.35 

d .  Predictor :  OAR 

Criterion:     Administrative  level  achieved. 

Correlation  coefficient   (r) :     0.44  (N=5,943) 

40.     Moses,  J.   L. ,   &  Boehm,  V.  R.      (1975).     Relationship  of 
assessment-center  performance  to  management  progress  of 
women.     Journal  of  Applied  Psychology,   60,  527-529. 

Purpose:     To  study  the  relationship  between  assessment  and 
subsequent  progress  in  management  for  women  in  the  Bell 
System. 

Type  organization:  Corporation. 
Subjects:     4,846  nonmanagement  women. 
Type  validity:  Predictive. 
Years  between  variables:     2  through  10 
Assessment  center?:  Yes. 
Range  restriction :     Low . 
Criterion  contamination?:  Yes. 
Results : 

a.     Predictor:     Aptitude  and  intelligence  measures. 
Criterion:     Administrative  level  achieved. 
Correlation  coefficient   (r) :  0.25 
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b.  Predictor;  Job-related  skills  indicators. 
Criterion:  Administrative  level  achieved. 
Correlation  coefficient   (r) ;  0.28 

c .  Predictor :  OAR 

Criterion :     Administrative  level  achieved. 
Correlation  coefficient   (r) ;  0.37 

41.  Nash,  A.  N.      (1966) .     Development  of  an  SVIB  key  for 
selecting  managers.     Journal  of  Applied  Psychology,   50,  250- 
254  . 

Purpose:  To  determine  if  the  measured  interests  of  a  large 
sample  of  managers  were  related  to  an  internal  criterion  of 
managerial  effectiveness. 

Type  organization:  Corporation. 

Subjects:     230  managers. 

Type  validity:  Concurrent. 

Years  between  variables:  0 

Assessment  center?:  No. 

Range  restriction:  High. 

Criterion  contamination?:  N/A 

Results: 

Predictor:     Psychological  attribute  indicators. 
Criterion:     Supervisor  ratings. 
Correlation  coefficient   (r) :  0.33 

42.  Prien,   E.   P.,   &  Liske,  R.  E .      (1962).     Assessments  of 
higher-level  personnel:     III.     Rating  criteria:  A 
comparative  analysis  of  supervisor  ratings  and  incumbent 
self-ratings  of  job  performance.     Personnel  Psvchol oav  15 
187-194.    — ' 

Purpose:     To  explore  the  relationship  between  supervisor 
ratings  of  job  performance  and  incumbent  self -ratings  of  job 
performance  on  tasks  which  are  intangible  in  nature. 
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Type  organization:  Corporation. 

Subjects:     96  supervisory  position  incumbents. 

Type  validity:  Concurrent. 

Years  between  variables:  0 

Assessment  center?:  No. 

Range  restriction:  High. 

Criterion  contamination?:  N/A 

Results: 

Predictor:  Self-appraisals. 

Criterion:     Supervisor  ratings. 

Correlation  coefficient   (r) :  0.25 

43.     Rankin,   K.  K.      (1981).     A  predictive  validity  study  of 
an  assessment  center  for  research  and  development  superiors. 
Wright-Patterson  AFB,  OH:  Air  Force  Institute  of  Technology. 

Purpose:     To  study  the  relationships  between  assessment 
center  ratings  and  several  measures  of  administrative 
achievement. 

Type  organization:  Military. 

Subjects:     67  civilian  nonmanagement  personnel. 

Type  validity:  Predictive. 

Years  between  variables:  6 

Assessment  center?:  Yes. 

Range  restriction:  Low. 

Criterion  contamination?:  Yes. 

Results : 

Predictor:  OAR 

Criterion:  Combination. 
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Correlation  coefficient   (r) :  0.00 

44.  Roadman,  H.  E.    (1964).     An  industrial  use  of  peer 
ratings.     Journal  of  Applied  Psychology,   48,  211-214. 

Purpose:  To  determine  if  peer  ratings  can  identify  managers 
who  later  are  promoted  rapidly  in  a  large  corporation. 

Type  organization:  Corporation. 

Subjects:     40  mid-level  managers. 

Type  validity:  Predictive. 

Years  between  variables:  2 

Assessment  center?:  No. 

Range  restriction:  High. 

Criterion  contamination?:  No. 

Results : 

Predictor:     Peer  ratings. 

Criterion:     Promotion  rate. 

Correlation  coefficient  (r) :  0.31. 

45.  Schmitt,  N.,  Noe,  R.  A.,  Meritt,  R. ,   &  Fitzgerald,  M. 
P.    (1984) .     Validity  of  assessment  center  ratings  for  the 
prediction  of  performance  ratings  and  school  climate  of 
school  administrators,  Journal  of  Applied  Psychology.  69, 
pp.  207-213. 

Purpose:  Evaluates  the  use  of  the  assessment  center 
approach  applied  in  an  educational  setting  to  select 
secondary  and  elementary  school  administrators. 

Type  organization:  Education. 

Subjects:     153  low-level  administrators. 

Type  validity:  Predictive. 

Years  between  variables:  1 

Assessment  center?:  Yes. 


Range  restriction :     Low . 
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Criterion  contamination?;  Yes. 
Results: 

Predictor :     OAR . 

Criterion:     Supervisor  ratings. 

Correlation  coefficient   (r) :  0.24. 

46.  Stahl,  M.  J.      (1983).     Achievement,  power,  and 
managerial  motivation:     Selecting  managerial  talent  with  the 
job  choice  exercise.     Personnel  Psychology,   36,  775-789. 

Purpose:     To  determine  if  high  managerial  motivation  is 
related  to  high  needs  for  achievement  and  power. 

Type  organization:  Corporation. 

Subjects:     45  mid-level  managers. 

Type  validity:  Concurrent. 

Years  between  variables:  0 

Assessment  center?:  No. 

Range  restriction:  High. 

Criterion  contamination?:  N/A 

Results : 

Predictor:     Psychological  attribute  indicators. 
Criterion:     Supervisor  ratings. 
Correlation  coefficient   (r) :  0.36 

47.  Steel,  R.   P.,   &  Ovalle,  N.   K.   2d.      (1984).  Self- 
appraisal  based  upon  supervisory  feedback.  Personnel 
Psychology,   37,  667-685. 

Purpose:     To  examine  the  relationship  between  self- 
appraisals  and  objective  criteria  of  job  performance. 

Type  organization:  Corporation. 

Subjects:     401  mid-level  managers. 

Type  validity:  Concurrent. 
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Years  between  variables;  0 
Assessment  center?:  Yes. 
Range  restriction;  High. 
Criterion  contamination?:  N/A 
Results: 

Predictor:  Self-appraisals. 

Criterion:     Supervisor  ratings. 

Correlation  coefficient   (r) :  0.19 

48.     Tenopyr,  M.   L. ,   &  Ruch,  W.  W.    (1965).     In  J.  P. 
Campbell,  M.   D.   Dunnette,   E.   E.   Lawler,   &  K.  E.  Weick 
(1970) .     Managerial  behavior,  performance,  and  effectiveness 
(p.   193).     New  York:     McGraw  Hill. 

Purpose:     To  study  the  relationship  between  personnel 
selection  measures  and  measures  of  job  achievement. 

Type  organization:  Corporation. 

Subjects:     113  mid-level  managers. 

Type  validity:  Concurrent. 

Years  between  variables:  0 

Assessment  center?:  No. 

Range  restriction:  High. 

Criterion  contamination?:  N/A 

Results: 

a.  Predictor:     Aptitude  and  intelligence  measures. 
Criterion:     Salary  level  attained. 
Correlation  coefficient   (r) :  0.29 

b.  Predictor:     Job-related  skills  indicators. 
Criterion:     Salary  level  attained. 
Correlation  coefficient   (r) :  0.36 
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49.  Thornton,  G.  C.     (1968).     The  relationship  between 
supervisory-  and  self -appraisals  of  executive  perf ormanace . 
Personnel  Psychology,    2 1 ,  441-455. 

Purpose:     To  investigate  the  relationship  between 
supervisory-perceptions  and  incumbent  self -percept ions  of 
the  performance  of  executive  personnel. 

Type  organization:  Corporation. 

Subjects:     64  high-level  administrators. 

Type  validity:  Concurrent. 

Years  between  variables:  0 

Assessment  center?:  No. 

Range  restriction:  High. 

Criterion  contamination?:  N/A 

Results : 

Predictor:  Self-appraisals. 

Criterion:     Supervisor  ratings. 

Correlation  coefficient  (r) :  0.24 

50.  Turnage,  J.  J.,   &  Muchinsky,   P.  M.      (1984).  A 
comparison  of  the  predictive  validity  of  assessment  center 
evaluations  versus  traditional  measures  in  forecasting 
supervisory  job  performance:     Interpretive  implications  of 
criterion  distortion  for  the  assessment  paradigm.  Journal 
of  Applied  Psychology,   69 ,  595-602. 

Purpose:     To  examine  the  ability  of  assessment  center 
evaluations  to  predict  actual  job  perfomance  criteria  and  to 
compare  the  predictability  of  assessment  center  evaluations 
versus  traditional  measures  in  forecasting  job  success. 

Type  organization:  Corporation. 

Subj ects :     799  mostly  nonmanagement  employees. 

Type  validity:  Predictive. 

Years  between  variables:     1  month  to  several  years. 
Assessment  center?:  Yes. 


Range  restriction :     Low . 
Criterion  contamination?:  Yes. 
Results : 

a.  Predictor:  Self-appraisals. 
Criterion:     Supervisor  ratings. 
Correlation  coefficient  (r):  0.01 

b.  Predictor:     Psychological  attribute  indicators. 
Criterion:     Supervisor  ratings. 
Correlation  coefficient  (r) :  0.01 

c.  Predictor:     Biographical  information. 
Criterion:     Supervisor  ratings. 
Correlation  coefficient   (r) :  0.09 

d.  Predictor:     Personal  interviews. 
Criterion:     Supervisor  ratings. 
Correlation  coefficient   (r):  0.03 

e.  Predictor:     Job-related  skills  indicators. 
Criterion:     Supervisor  ratings. 
Correlation  coefficient  (r) :  0.02 

f.  Predictor:     Peer  ratings. 
Criterion:     Supervisor  ratings. 
Correlation  coefficient  (r) :  0.08 

g .  Predictor :  OAR 

Criterion:     Supervisor  ratings. 
Correlation  coefficient  (r) :  0.03 

h.  Predictor:  Self-appraisals. 
Criterion:     Salary  level  attained. 
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Correlation  coefficient   (r):  -0.01 
i.     Predictor:     Psychological  attribute  indicators. 

Criterion:     Salary  level  attained. 

Correlation  coefficient   (r):  -0.02 
j.     Predictor:     Biographical  information. 

Criterion:     Salary  level  attained. 

Correlation  coefficient  (r):  0.05 
k.     Predictor:     Personal  interviews. 

Criterion:     Salary  level  attained. 

Correlation  coefficient   (r):  0.10 

1.     Predictor:     Job-related  skills  indicators. 

Criterion:  Salary  level  attained. 
Correlation  coefficient   (r):  0.07 

m.     Predictor:     Peer  ratings. 

Criterion:     Salary  level  attained. 

Correlation  coefficient   (r) :  0.04 
n .     Predictor :  OAR 

Criterion:     Salary  level  attained. 

Correlation  coefficient   (r) :  0.12 

51.     Williams,   F.  J.,   &  Harrell,  T.  W.      (1964).  Predicting 
success  in  business.     Journal  of  Applied  Psychology.  48, 
164-167. 

Purpose:     To  identify  those  predictors  which  are 
significantly  correlated  with  on-the-job  achievement. 
Type  organization:  Corporation. 

Subjects:     19  6  MBA  graduates. 

Type  validity:  Predictive. 

Years  between  variables:  15 

Assessment  center?:  No. 
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Range  restriction :     Low . 
Criterion  contamination?:  Yes. 
Results: 

a.  Predictor:     Aptitude  and  intelligence  measures. 
Criterion:     Salary  level  attained. 
Correlation  coefficient   (r) :  -0.01 

b.  Predictor:     Aptitude  and  intelligence  measures. 
Criterion:     Adminstrative  level  achieved. 
Correlation  coefficient  (r):     -0.01  (N=116) 

c.  Predictor:     Psychological  attribute  indicators. 
Criterion:     Salary  level  attained. 
Correlation  coefficient   (r):  0.08 

d.  Predictor:     Psychological  attribute  indicators. 
Criterion:     Administrative  level  achieved. 
Correlation  coefficient   (r) :     0.09  (N=116) 

e.  Predictor:     Biographical  information. 
Criterion:     Salary  level  attained. 
Correlation  coefficient  (r) :  0.17 

f.  Predictor:     Biographical  information. 

Criterion:     Administrative  level  achieved. 

Correlation  coefficient  fr) :     0.11  (N=116) 

52.     Wollowick,   H.   B. ,   &  McNamara,  W.  J.  (1969). 
Relationship  of  the  components  of  an  assessment  center  to 
management  success.  Journal  of  Applied  Psychology.   53. ,  348- 
352. 

Purpose:     To  determine  the  validity  of  an  assessment  center 
approach  in  predicting  management  potential  and  to  determine 
the  relative  value  of  the  components  of  the  program. 


Type  organization:  Corporation. 

Subjects:     94  low-  and  mid-level  management  males. 

Type  validity:  Predictive. 

Years  between  variables:     2.5  through  3.2 

Assessment  center?:  Yes. 

Range  restriction:  High. 

Criterion  contamination?:  Yes. 

Results : 

a.  Predictor:     Aptitude  and  intelligence  measures. 
Criterion:     Administrative  level  achieved. 
Correlation  coefficient  (r) :  0.09 

b.  Predictor:  Job-related  skills  indicators. 
Criterion:  Administrative  level  achieved. 
Correlation  coefficient  (r):  0.20 

c .  Predictor :  OAR 

Criterion:     Administrative  level  achieved. 
Correlation  coefficient  (r) :  0.37 
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